Best Practices for Allocations

So, you’ve procured a supercomputer—a product of a consortium of stakeholders whose goal is to combine resources to fulfill one or more mission objectives. And you have a constant stream of applicants who would like to use its cycles. The stakeholders have appointed an allocation committee to periodically review applications for time on the computer and decide which new projects to accept, which ones to extend, and which ones to end. This process is repeated at regular intervals, whether annually, semi-annually, quarterly or monthly. A part of the allocation committee’s responsibility is to decide is how many processor-hours to permit each project to use within the period.

In support of this need, Moab Accounting Manager (MAM) supports the concept of allocations. An allocation is a pool of resource credits that is given to a project for use within a specific time period. A project’s members can run jobs on the supercomputer, each job charging against their allocation, until they run out of funds. When a project reaches the end of one allocation period, the allocation may be extended by resetting the allocation to the full amount of the next period. As a period progresses, users need to know how much of their allocation they have remaining.

There has been a progression of methodologies used to manage allocations in MAM—varying in the way that we create the allocations and in the way that we reset them. In this post, we will discuss the methodologies that have been used in the past and give recommendations for the best practices for today. We will discuss methodologies that we will call expiring allocations, resetting allocations and discrete allocations.

Allocations

Expiring Allocations

Under this methodology, allocations are created ahead of time, with a start time and an end time that are both in the future. For example, before the beginning of the year you might create four quarterly allocations for project 1, one that starts at the beginning of the first quarter and ends at the end of the first quarter, another that starts at the beginning of the second quarter and ends at the end of the second quarter, and so on. When the beginning of the year comes around, the first allocation goes into effect, with the other three allocations remaining dormant. At the end of the first quarter, the first allocation and its remaining budget becomes expired, and the disbursement for the second quarter becomes active. This was a good method, but one of its weaknesses included the fact that there was a lot of administration required to manage multiple allocations for each project. This methodology was prone to mistakes and often resulted in overlapping allocations. With the potential for overlapping allocations, it became difficult to say what percent of your allocation had been used.

Resetting Allocations

These circumstances led to a new strategy of reusing a single allocation for each project. Instead of having multiple allocations per project, a scheduled task would launch a script at the end of each period that would start, reset or end each project allocation according to the project disbursements decided by the allocation committee. The script would have to be adjusted or a data source updated to enforce the new allocation information. This practice would eliminate the potential for overlapping allocations and reduce the administration required. However, it became more difficult to report on current and past allocation usage in terms of percent used and unused portions because the usage was all blended into a single allocation—with no clear end and beginning times. And there was the need for the administrator to continually maintain and update a custom script to key in the changes that should occur at the beginning of the next allocation period.

Discrete Allocations

The challenges that arose in the preceding methodologies elicited the deployment of a new methodology called discrete allocations. When using discrete allocations, Moab Accounting Manager attempts to ensure that allocations are non-overlapping (in time) and non-reusable (each allocation period should use a distinct allocation). This behavior is designated by the allocation.enforcediscrete server configuration parameter and is the default in the new release.

Allocations have been enhanced to internally track the initial allocated amount, the current allocated amount and the remaining amount within the allocation object itself. Each time that an allocation is reset for a new period, a separate (self-tracking) allocation is automatically created.

A new default deposit amount field can be set for each project (assuming a single project per fund). The default deposit amount is used when an allocation is reset. If the default deposit amount is positive, the currently active allocation is ended and a new allocation is created with the default amount. If the default deposit amount is set to a value of zero, the active allocation is ended and no new allocation is created. If the default deposit amount is not set, the allocation is not affected. The reset can be performed via a scheduled event or via a cron script. If default deposit amounts are kept up-to-date (including being zeroed out for allocations that are slated to end and being unset for allocations that you do not want affected by the reset), automation of this method can be as simple as creating a single periodic event with a FireCommand of “Fund Reset.”

Thus, by using discrete allocations, administrators can eliminate the need to manage a complicated script or multiple allocations per project. It is necessary only to edit the default amount for each fund prior to the allocation period boundary in which it must change. Since allocations are prevented from overlapping and since allocations remain distinct for each period, the reporting of aggregate allocation information has become unambiguous and trivially easy to extract.

Conclusion

Today, we recommend the use of discrete allocations to avoid the undesirable effects of overlapping and reusable allocations. We recommend the use of a periodic event in MAM to reset all applicable allocations on the allocation cycle. Administrators are encouraged to utilize the default deposit amounts in order to start, change or end the amounts allocated to projects for the next period. Using this methodology should result in less work, fewer mistakes and more intelligible reporting.

Facebook Twitter Email

Speak Your Mind

*