I help engineers be more productive and compete with bigger firms by building infrastructure to process data quickly.
I have a background in big data, from my work in the genealogy field. A few years back I was talking with some engineer friends, a common thread in the conversation was how much time was spent waiting on computers to process the data required to make their assessments.
I realized the process I was using to distribute family history workloads across multiple computers, would also work for say pipeline defect analysis. I could break the data down to individual defects, distribute that defect data to one of many computers, and run it through the equations required to determine the life span of the defect. This process works for a multitude of problems, not just pipeline defect analysis.
In this manner, a data set can be processed very quickly. Allowing the engineer to analyze the data much quicker than if waiting on more traditional methods to process the data. This also has value in helping the engineer react to changes in requirements, if you can process data faster and a change to the model being used is requested it is less impactful because the data is processed so much quicker.
I am not an engineer though (my degree is in computer science), I build and manage the infrastructure to distribute the data to the many computers. And I work closely with the engineer to program the equations needed and the tests to prove the equations are correct.
This is the first step. We work together to break your workflow down to the smallest units of work. And setup a plan to process your data.
After the plan is put together I develop the software. Reliability and reproducibility are imperative at this point, so tests always accompany.
The cloud isn't just some one elses computer, it is an affordable way to get high performance computing without a high price tag.