Salesforce Einstein Vision gives you a set of powerful API's that can be used to bring image recognition capabilities into your CRM application. Image recognition can be used for Visual search, Brand detection, Product identification and many more. With Salesforce Einstein Vision, you can leverage pre-trained image classifiers, or train your own custom classifiers to solve a vast array of image-recognition use cases, empowering end users across sales, service, and marketing to be smarter and more predictive.
What is Feedback loop?
A feedback loop refers to the process by which an AI model’s predicted outputs are reused to train new versions of the model. Salesforce Einstein Vision Platform provides set of powerful APIs for adding feedback loop which improves the image classifier's prediction capabilities by using the model's misclassified images to train new versions of the model.
Feedback loops ensure that AI results do not stagnate. This also has a significant advantage as the data used to train new versions of the model is of the same real-world distribution that the customer cares about predicting over. Without it, AI will choose the path of least resistance, even when that path is wrong, causing its performance to deteriorate. By incorporating a feedback, you can reinforce your model's training and keep improving them over time.
Completing the Feedback loops with Einstein Vision
In many applications developer train their AI models with sets containing large number of labeled image data. Over the time, the prediction accuracy might decline for some set of images.
Luckily, Einstein Vision APIs provides a way to limit this issue and keep your prediction results accurate everytime.
Feedback loop with Einstein Vision APIs can be used in 3 simple steps
The model returns a response that contains a high probability that the image is a beach (in the Beaches class). However, you expect a response that contains a high probability that the image is a mountain (in the Mountains class). This means that the image was misclassified.
This misclassified image can be used to provide feedback to the model in order to increase accuracy for the future predictions.
How it's done?
Before we start, if this is the first time you are going use and explore Einstein platform APIs, I would recommend you to go through the Trailhead modules to gain basic knowledge on Einstein. Once you have some basic knowledge you can start with Einstein Vision and Language Model Builder app, created by Salesforce Labs, is available for free on AppExchange. This app is super helpful for quickly uploading dataset, training models and predicting the results. Follow the instructions in the Quip doc.
Once you are familiar with the basic Einstein Vision APIs, you can refine your model by using below feedback APIs
Step 1:
The first step is to add the misclassified images to the dataset along with their correct labels.
Step 2:
After we add feedback examples to a dataset, we can query the dataset and return just those examples that were added from the feedback API call.
Step 3:
Once we retrieve and validate the feedback images we have 2 options to train our model
Additional request parameter trainParams={"withFeedback" : true} is used with retrain endpoint.
![](https://1.bp.blogspot.com/-k4lDYCgZUOc/XaQVyDYB2WI/AAAAAAAAK7I/lTz5AJ6e-QUhYPyB1sJBBNWOQ4_D5fMrgCLcBGAsYHQ/s640/After2.PNG)
Other Resource: Get Started With Building Einstein Vision and Language Models
Trailhead Project: Quick Start: Einstein Image Classification
Why we need Feedback loop?
In today's world every industry uses feedback loops to streamline their operations. In business, the practice of taking customer feedback (the output of a product or service) and using it to improve future processes is best example and commonly used feedback loop.Feedback loops ensure that AI results do not stagnate. This also has a significant advantage as the data used to train new versions of the model is of the same real-world distribution that the customer cares about predicting over. Without it, AI will choose the path of least resistance, even when that path is wrong, causing its performance to deteriorate. By incorporating a feedback, you can reinforce your model's training and keep improving them over time.
In many applications developer train their AI models with sets containing large number of labeled image data. Over the time, the prediction accuracy might decline for some set of images.
Luckily, Einstein Vision APIs provides a way to limit this issue and keep your prediction results accurate everytime.
Feedback loop with Einstein Vision APIs can be used in 3 simple steps
- Add a misclassified image to a dataset with the correct label.
- Get a list of images that were added as feedback to a dataset.
- Create a model or update an existing model using feedback images.
The model returns a response that contains a high probability that the image is a beach (in the Beaches class). However, you expect a response that contains a high probability that the image is a mountain (in the Mountains class). This means that the image was misclassified.
![]() |
LakeSideMontain.jpg |
How it's done?
Before we start, if this is the first time you are going use and explore Einstein platform APIs, I would recommend you to go through the Trailhead modules to gain basic knowledge on Einstein. Once you have some basic knowledge you can start with Einstein Vision and Language Model Builder app, created by Salesforce Labs, is available for free on AppExchange. This app is super helpful for quickly uploading dataset, training models and predicting the results. Follow the instructions in the Quip doc.
Once you are familiar with the basic Einstein Vision APIs, you can refine your model by using below feedback APIs
Step 1:
The first step is to add the misclassified images to the dataset along with their correct labels.
curl -X POST -H "Authorization: Bearer <TOKEN>" -H "Cache-Control: no-cache" -H "Content-Type: multipart/form-data" -F "modelId=<MODELID>" -F "data=@c:\data\LakeSideMontain.jpg" -F "expectedLabel=Mountains" https://api.einstein.ai/v2/vision/feedback
Step 2:
After we add feedback examples to a dataset, we can query the dataset and return just those examples that were added from the feedback API call.
curl -X GET -H "Authorization: Bearer <TOKEN>" -H "Cache-Control: no-cache" https://api.einstein.ai/v2/vision/datasets/<DATASET_ID>/examples?source=feedback
Note: Additional source parameter is used to query only feedback images Step 3:
Once we retrieve and validate the feedback images we have 2 options to train our model
- Create new Model
- Update existing Model
Additional request parameter trainParams={"withFeedback" : true} is used with retrain endpoint.
curl -X GET -H "Authorization: Bearer <TOKEN>" -H "Cache-Control: no-cache" -H "Content-Type: multipart/form-data" -F "modelId=<MODELID>" -F "trainParams={\"withFeedback\" : true}" https://api.einstein.ai/v2/vision/retrain
Additional Resources
Documentation: Einstein Platform Services Developer GuideOther Resource: Get Started With Building Einstein Vision and Language Models
Trailhead Project: Quick Start: Einstein Image Classification
Comments
Post a Comment