Adaptive for Bridging Small and Big Data
“A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: a complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a simple system.” – John Gall, systems theorist
Structured, semi-structured, unstructured, Linked, Cloud, Internet, Farm, Lake – terms associated with traditional and non-traditional forms of data types and access that together bridge nearly every type of IT architecture and data management process over the past 40 years. With each passing year, new technologies and methodologies hold the promise of unlocking new levels of business value. Today, there are over 70 vendors in the Big Data space alone and growing. Not only is the system complex, it’s getting more complex each day. For all its variations, the data landscape can be classified in one of two ways- Small Data and Big Data. Small Data is data in which we have a sense of where it is coming from and how much there will be. For example, companies large and small know their customer base and can design database systems to accommodate this data. Big Data is data from sources for which we have no way to estimate how large they will be, how much they will grow and how much they will change. Organizations have made investments in Small Data for years and are now embracing the promise of Big Data. However, the governance, rules and processes are different with Big Data. The need exists to bridge Small and Big Data by leveraging the knowledge embedded in existing simple and complex systems, present it in a common understandable format and provide collaborative governance from which new solutions can leverage, measure and guide – that capability is here today.
Big Data Semantics and Knowledge Governance.
For over a decade Adaptive has embraced the key tenets of visibility, agility and governance for all stakeholders through best in class focus on data traceability and lineage. Adaptive believes that in order to successfully govern and navigate the operational complexities of any system (simple or complex) organizations must have a strong understanding of how their metadata enables visibility. Clear visibility of current state drives successful implementation of new paradigms. Without question, today’s data enablers will be tomorrow’s legacy components. The cost of rewriting existing processes, hiring on new classes of stewards, purchase of new tools and technologies cannot be a yearly thing. But metadata is just one component. The ability to understand the meaning of data, drive new data meaning and create ‘knowledge bridging’ requires multiple components leveraging metadata at the core. This is required not only for the purpose of building new levels of insight but also providing the foundation for a new class of knowledge governance enabling future change agility independent of the underlying system. Adaptive views key tenets such as ontologies, semantics and rules based decision support as being ‘metadata complementary’, providing the ability to both semantically link IT with the business and further, link the business with industry compliancy / regulation / internal standards.
The Data Ontologist.
Adaptive is the only vendor that offers an integrated metadata and semantic approach ensuring system independence while retaining and expanding upon the most important corporate asset of all, its knowledge. This knowledge is best represented through structural frameworks describing data and their relationships, quite simply, ontologies. Ontologies are structural frameworks consisting of relationships from which meaning can be derived. They are constructed with a common vocabulary given pre-described sets of rules.
With Adaptive, Ontologies become the core component leveraging our heritage of metadata success in stitching, mapping, classifying, tagging, rules and transformations. In addition, Adaptive’s Ontologies are built on standards such as OWL and RDF allowing for standards based integration with third party Semantic Web components. Ontologies can be created and extended either manually or automatically by applying rules to map from existing logical models or data sources, each containing their own sets of relationships. Thus, by simply leveraging the metadata within either a simple or complex system, knowledge is derived and extended as the ongoing data lifecycle repeats. Adaptive has coined a new type of steward called the ‘Data Ontologist’ responsible for managing this process.
Adaptive for Big Data.
Adaptive today helps both firms and governments bridge Small and Big data by placing our metadata management repository at the heart of their operations. Our own thought leaders and consulting partners can help you design a smart and structured approach that will enable you to leverage your existing solutions and technology investments which you may already have introduced as part of your efforts, and comfortably achieve insight into and integration with your Big Data.