Jan 2018 to Present
I built a data lake for Cansure. At the same time I was volunteering for the City of New Westminster on their Open Data. The City of New West expressed frustration about the number of disparate data sources (taxes, business licenses, dog licenses, building permits, etc, etc, etc). They also expressed frustration that it takes a day or more to “anonymize” data before making it openly available. The Open Data files are then hosted as flat, text files on their website server.
The challenges New West staff expressed sounded awfully similar to the challenges Cansure expressed, and before that Optimum Insurance. I believe all businesses need to pull together disparate data, make it easily accessible, query via APIs, and manage permissions/security.
I pulled together a friend and my wife to be my 2 co-founders. I specifically wanted female influence on the business because I’ve found my wife and our friend Jen make far more sound decisions than I do. I also wanted the two co-founders to be able to out-vote me if they feel I’m making a bad decision.
First, a key benefit of Lakebed is it can be hosted on-premises or on a cloud server. This allows a data/software developer to run a local, development copy of the data lake. The code, reports, dashboards, apps, etc. can then easily be pushed to production and utilize the production data lake.
Second, Lakebed is the “WordPress of data lakes”. It can be installed in less than a day and then populated by simply dragging & dropping Excel data files. An organization might hire a data warehouse developer who would cost $100,000 annually and take at least a month to build a data lake/warehouse. Or, an organization can buy a Lakebed license at a fraction of the cost and have it running in 1-2 days. And, that license entitles the customer to ongoing updates to ensure the code & database are always up-to-date.
Here’s a video I made explaining what a data lake is, what Lakebed does, and how quick it is to use: