A Compostable Composable Platform

An easy autocorrect misunderstanding, but something I couldn’t shake off as an analogy for the platform we need in today’s changing world. A platform that ensures each investments in applications, data and AI, should be built from the foundational building blocks of our unique organizations, our own data. We’ve all been pointing that way for some time, but perhaps, if you’re like me, not really able to visualize how we get to this apparent Nirvana. So let’s take a walk through the journey we’ve all followed (some steps of it at least as there have been many branches) and why this analogy fits so well as the logical next step.

Pre-history

Cave drawings, papyrus, slates, paper, punchcards, magnetic tapes etc… Life’s too short and tbf mostly before my time.

The push into the big blue(s)

The mainframes and ERPs, arguably the first mass migration into systems trying to capture the business processes and data of the organization. To their credit they had the vision to try and model the data in a holistic manner, but of course this first foray had its complications and limitations. I personally remember, some 30 years later, a number of those shortcut codes to navigate around the first few mainframe and ERPs I worked on. UI’s certainly improved but shortcut codes remained the expert navigation of choice for these systems. AC16 anyone?

Craving to see

Building systems around processes and data captured a lot, but made very little truly visible outside of the management of the single record/case itself. So the next phase I experienced was the desperate desire to be able to see that data. To report, to aggregate, to be able to make decisions from it. The same systems that captured the data struggled to evolve to meet this need. Excel bounded onto the scene and made our lives way more interesting. We’ve all seen the best/worst of excel models. My personal masterpiece/monster had about 70 linked files (all many linked sheets deep) which resolved actuals and planning for all levels of reasonable size, and reasonably famous, organization; from roles to teams, to departments, to business units and finally to a full group level roll up. It was amazing as an exercise in how far you can push things, but there was obviously a better, and less error prone, way.

The Data Warehouse epoch

What self-respecting organization has not plunged into the crevasse of the data warehouse project. Don’t get me wrong, I loved designing graceful data transformations and modeling celestial star schemas, and making big advances to reporting and analysis in the process. This was a great period to be a data ‘nerd’, those who could see the relationships in the data now had the ability to let others see the same. But, the downside of such capable depth was that it took a huge amount of time to do well and maintenance a significant burden and resource drain.

Dump it in the lake

Let’s assume the tools are smart enough and just dump all this data into one place. Let’s not even structure it as surely the tech is good enough to make sense of it, right? It’s a great idea, perhaps it was a little before its time to meet the expectations the marketing had you believe. But all these steps bring us to this new world we find ourselves with the promise of answers to everything without putting in any effort at all, hurrah! Well, almost.

I want it all, I want it all, I want it all, and I want it now!

Large language models (LLMs, as if you needed me to add that), the supernova that has eclipsed everything before, at least as far as user expectations are concerned. So are we now coming to the era where we can have everything we always dreamed of? Unfortunately, not if you want it to grasp your specific organization’s challenges, and not without the expense, effort and risk of some very significant projects/programs along the way… But that’s where the compostable analogy kicks in, a new paradigm perhaps.

Composable applications, compostable data!

Imagine your data managed holistically, your applications integrated seamlessly, no silos and vendor constraints, enabling the capture of structured and unstructured data as needed, a true composable platform for your Organization.

Now imagine a swarm, a fleet, a gaggle, a pride of bots (what is that collective noun?) tailored for your organization, always processing as data changes, ‘breaking down’ that data into key nutrients (not insights yet, as they come with application) continually evolving and adjusting their learned outputs, adding to your nutrient library. Let the data be ‘decomposed’ (not the original data of course) for your organization, your context, your processes to give your organization the nutrients it needs to grow, to be guided with true context, to meet it’s ultimate potential.

Now you have the base, a real base, for a supercharged RAG approach informed and augmented by the core nutrients of your organizations learning. Continuously evolving and tuning itself for your organization, and never leaving the confines of your data boundaries. And all this running on a composable platform where all of the pieces of the orchestra (the building blocks) are themselves evolving and improving whilst maintaining and improving your apps, your agentic workflows, your composed views and layouts. Nothing stationary, everything incrementally improving every week whilst maintaining the integrity of your processes…

Now that’s an application and AI strategy I can totally get behind!

Credits: Kudos to Adrian Parker at Differentia Consulting who triggered this little moment of mind wandering about the nature of a compostable approach to data and AI.