thumbnail

The best practices in releasing new versions for your Machine Learning APIs

As new sets of data come in, it is necessary that you refit and release new versions of your machine learning models. In this article, we share the best practices and tips that will help you go through the release process smoothly.

As new sets of data come in, it is necessary that you refit and release new versions of your machine learning models. In this article, we share the best practices and tips that will help you go through the release process smoothly.

1. Release Gradually (A/B Testing)

When you have an existing version, ensure a new version processes only a small portion of transactions in the beginning. Even though the new version is thoroughly tested, things can go wrong in a production environment and we do not want our customers to have bad customer experiences.

Another benefit of releasing gradually is that you can see the added value of the new version in a real environment. Simply because a version is “new,” it is not necessarily better. You can let two versions (existing and new) fight each other for some time to observe which performs better and then disregard the new one if it performs worse. You can even further research to see if there are segments that one version performs better than the other and use your findings to improve overall predictions.

2. Prepare a Revert Process

Always expect the worst. A version might work well in a small-scale staging environment. However, whether it can handle transactions fast enough to support a large volume of requests in a production environment is another question. Release when you have enough developers and development operation engineers to monitor and take actions if something goes wrong. Consider releasing when a number of transactions is small so that the side-effect of a release (if any) is minimal. Lastly, automate the revert process so that when a revert is necessary, it can happen immediately, precisely and fast.

3. Know the Differences

Releasing a new version has not only technical but also legal and business implications. The IT team needs to understand how a new version runs differently from an existing version to ensure whether a new version might require a better infrastructure (e.g. a Deep Learning model might require servers with GPU), a database is ready to provide any new variable that the version will require, and the new request processing time won’t be too longer than the current level. Business owners need to ensure that the new version does not use logic nor variables that can impose legal implications. Comparing code bases of versions can be a good and easy way to understand the implications.

4. Approval Process

An organization needs a systematic way to review, approve and release a new version to ensure that technically the new version is sound, and all business-related implications are communicated to managers. The process can be from a simple verbal communication whenever a new version is released to an application that controls access management of users in various roles and facilitates communication and approval requests among them.

With Knowru, you can incorporate these best practices to your organization from Day 1. Knowru comes with these features out-of-box.

1. Release Gradually (A/B Testing)

You can test your different runnable versions and release gradually on Knowru. To release your new version while running an existing version simultaneously, go to your machine learning API’s detail information page and select the “Adjust Weights” button. Enter small numeric values as a weight for your new version while having a large value for the existing version. A version’s weight echoes the portion of transactions that will go through that version.

2. Prepare a Revert Process

Knowru’s Adjust Weight feature which we just explained can work as a Revert Process as well. Open your API’s detail informationpage and click the “Adjust Weights” button. Enter the previous weight values to revert to the situation before a release.

3. Know the Differences

You can track the changes and get the differences between your previous and current runnable versions using Knowru. Simply go to one of your versions’ detail pages. Click the “Get Diff” button. It will show (+) and (-) in the beginning of each line for an addition and removal respectively.

4. Approval Process

On top of all these features, Knowru provides a way to systematically communicate and approve releases of new versions. First of all, a manager can designate an API to be “In Production”, which will limit a member from updating weights (thus releasing) of versions without an approval from a manager.

When the member requests to adjust weights of an API, all managers will promptly receive an email with a link a page showing all the details the manager need to know to make an approval decision. Once a manager decides, Knowru will notify the decision to all other managers and the adjust weight requester via email for prompt communication.

Tags

Comments