Introducing two features to control token consumption for PFP inference: Max Consumption Percentage (PFP Rate limit) and a similar feature, Job Priority.
- Max Consumption Percentage (PFP Rate limit)
- Priority
- Feature Comparison
You may also refer to the Matlantis Guidebook for more information.
Max Consumption Percentage (PFP Rate limit)
Overview
- An administrator can limit the available token consumption rate for each user. This rate is calculated as a percentage of the token recovery rate allocated to the tenant, with the tenant's full recovery rate being 100%.
- You can configure this setting in Dashboard > Users.
- The settings are applied immediately upon an update, even while Matlantis is running (no restart is required).
- Theoretically, if the total consumption percentage for all users within a tenant is set to exactly 100%, the Token Availability will not drop below 100% (i.e., it will not "burst").
Image of PFP Load Status Token Consumption Rate Plot Graph:
When multiple users run calculations that reach their consumption limit
- The stacked graph will appear as a flat line, sticking to the maximum consumption rates you've set.
- The rate allocated to each user will be displayed in the PFP Load Status as "Your Max Rate: xx%".
Usage Example
- This feature allows administrators to set the amount of tokens available to each user based on their department, which is useful when a single tenant is shared by users from different teams.
Priority
Overview
- You can set the priority of a job to control its execution. When Token Availability decreases, jobs are executed based on their priority setting, which is a number from 1 to 100. A higher number indicates a higher priority.
- Example 100 (Default): The job runs regardless of the remaining token amount.
- Example 30: If Token Availability drops below 70%, the job will be put on hold until it recovers to 70% or more.
- There are several ways to configure priority. For more details on these methods, please refer to the corresponding documentation.
- If multiple priority settings are configured, the one with the highest precedence will be used. The following lists the settings in order of precedence, from strongest to weakest:
- As an argument to the Estimator (constructor)
- Assigned to the Config.priority variable
- Set as an environment variable
- Set when executing a background job
Usage Example
- As a general proctice, setting a low priority allows you to ensure that urgent, high-priority calculations are executed first when they are needed.
- You can set a lower priority for non-urgent, time-consuming jobs. This is a considerate approach to avoid negatively impacting the calculations of other users.
Feature Comparison
| Max Comsumption Percentage （PFP Rate limit) |
Priority | |
| Users | admin users | All Notebook Users |
| Configuration Method | Dashboard > Users *Users without admin privileges cannot set this. |
Multiple configuration methods exist (see above) |
| Applicable Scope | Per user (for all of that user's calculations) |
Depends on the configuration method.
|
| Applicable Timing | Always | When Token Availability drops below 100%, depending on the set value. |
FAQ
Q: Can Max Consumption Percentage and Priority be set at the same time?
A: Yes, they work together without any issues.
...
Appendix:
Setting Priority for Background Jobs
On the background job execution screen, you can enable Use a lower Request Priority for this Background Job and enter a numerical value. The default value of 10 means the calculation will be put on hold if Token Availability is not at 90% or higher.
(Reference) How to use Background Job > Execution
https://redirect.matlantis.com/api/docs/en/matlantis-guidebook/about-background-job.html#execution
Executing via the Terminal
env PFP_DEFAULT_PRIORITY=10 mtl-bg-job run phonon.ipynb phonon_results.ipynb
(Reference) How to use Background Job > Command Line Tool