Some might argue that “Data Should Be Free”. We could debate that, but one thing that is NOT up for debate is “Compute Power is NOT FREE”.

A lot of tech companies have two sides to their revenue stream: First, the customer facing side where they fulfill a customer's need and second, the 3rd party vendors they share data gathered from the customers with.

Before we go too far I am not here to debate the ethics of sharing data. It is happening, businesses are selling the data. Personally I suggest you take your privacy seriously but that is a post for another time.

Whether you charge for your data or not is up to you. What you have to charge for at a certain point is compute power. Calculating, querying, and correlating data all requires cloud CPUs running and costing someone money. That can either be you or it can be your big data consumer customer.

A common downfall I see in the tech industry is that companies design the customer facing product really cleanly but the big data consumer facing product is just duct taped together. I suggest you be very mindful of what that product looks like and design it in a way that prevents them from abusing your system.

One solid design rule I like to follow is: Nothing the data consumers do should be able to negatively affect your regular customers. How do I do this? Never let data consumers query your source of truth DB. They should always be hitting data warehouse/data lake/data (insert other buzz word here). The nice thing here is if you use a serverless big data solution like AWS Glue then you can easily calculate what costs were incurred by which consumer, then mark that up and send them a bill for it.

The bottom line is: Compute Power is NOT free. Do not get stuck with a massive AWS bill because you didn’t design those costs into the big data consumer side of your tech product.

~Cheers

PS: If you are interested in engaging my services as a Web Application Architect check out my Group Coaching Program or 1 on 1 Consulting at schematical.com and check out my FREE eBook 20 Things You Can DoTo Save Money On Your Amazon Web Services Bill Today