Accuracy Improvement Challenge
You already have a data science or artificial intelligence team, and you already have a very important model. Your model might be anything from fraud-detection to text classification, through to image segmentation, pattern recognition, or even just basic sales forecasting. You already have a data set, and infrastructure for testing new models. Your current R&D process is not producing much improvement, or may be bogged down with other work. Your looking for fresh ideas for improving accuracy, and the help to implement those ideas. But you don't want to risk a lot of money on R&D that might go nowhere.
Bring your models to Electric Brain and challenge us to improve the accuracy. You set your own price and pay only for the improvement in performance that we are able to obtain on your model. Set your own "bounty" so to speak, and we will test new ideas and approaches until we believe there is no more juice we can squeeze out your model for the amount of bounty you offer. Set your own price and only pay for performance. We then work our magic.
Before we commit to making any contingency based deal, we must know a great deal about the model, your existing R&D process, your dataset, etc.. to make a judgement on whether we can actually improve results and your bounty is worth it.
What does the model do? What is the business value of the model?
What programming language is your current models written in?
How do you measure the accuracy of the model?
What is your evaluation metric?
How do you trade off between different characteristics of the model, such as false positives vs false negatives?
How large is your dataset? Provide exact number not rough estimate.
How was your dataset annotated / labeled?
Who does the labeling?
What is the error rate of human labeling?
How many people do you have researching this model already?
What approaches has your research team tested already? What is the accuracy of those approaches?
Orientation and access to data
Initially, we just need to get oriented around your data and get direct access to the databases if we can. This may involve getting authorization from your security department. It may also involve sitting down with your database engineers to go over the database structure, with a specific focus on the structure of the tables in question.
If the datasets that you are working with are so large that they need to be processed using special distributed tools, then we will need to get ourselves oriented around your data-processing infrastructure as well.
Obtaining lots of computer power
Now with proper access to your data, we need to obtain access to the compute power that we can use to run experiments. This is important - we are an compute heavy data firm. Our ability to crack any model and improve accuracy may seem magical, but it just comes down to raw, rapid paced experimentation. It means trying many different things, in parallel and each of those things usually involves grid-searching parameters in part of the process.. We therefore ask for as much compute power as your budget can allow for. We will reject projects when provided with only a small compute budget. If your unsure how much to allocate, we can find a way to bake the compute costs into the bounty and we will allocate our time and compute power according to what we think is best.
Time & Cost
You are flexible to set your own bounty for the project, and we will determine whether its worth our effort to take on the project. For obvious reasons, projects with a larger bounty will get a larger amount of effort from us. Additionally, we reserve the right to cancel the project once we determine that any additional use of our time is not worth your provided bounty. We will provide a couple quick examples of bounties.
Low Hanging Fruit
You have a model that has not been researched much and is just needing some tender love and care that you are unable to provide with your current team. The model is a low hanging fruit, with lots of potential for improvement.
Existing Accuracy: 85%
Target Accuracy: 90%
Bounty: $10, 000 per 1% improvement plus $40, 000 bonus if 90% is reached.
Computer power: Paid for by client, lots available
Time: 4 weeks to 8 weeks
A Deep Learning Upgrade
You have a model that has been around for a while and does pretty well with classic techniques. You are interested in finding out if upgrading your model to use deep-learning will improve results. You have a very large dataset - you just need expertise in neural networks, without taking a lot of risk.
Existing Accuracy: 80%
Bounty: $15, 000 per 1% improvement
Computer power: Paid for by Electric Brain
Time: 8 weeks to 12 weeks
A Long Running Challenge
You already have a model that solves an important business problem. The model has existed for a long time and has been researched to death by your existing team. It will be challenging to improve the performance, but even the smallest improvement of accuracy means a huge boost to your companies bottom line. Its challenging and will demand a large bounty, but its worth the price to get the accuracy improved.
Existing Accuracy: 90%
Bounty: $50, 000 per 1% improvement
Computer power: Paid for by client, lots available
Time: 16 weeks to 24 weeks