The Whiteboard

Semantic Arts future-proofs the enterprise with data-centric transformation programs.

99% of all enterprises shoot themselves in the foot every time they implement a new information system. The prevailing mindset, the “application-centric” mindset,  guarantees that each new system introduces yet another incompatible data model into the firms datascape. It doesn’t matter whether the firm is building a custom application, buying an application package, or renting Software as a Service. It doesn’t matter whether the methodology is waterfall or agile. Solving business problems by implementing applications is what creates the dreaded silos.

The compounding problem is these application systems each have their own completely idiosyncratic data models. In addition to being arbitrarily different, almost every application is at least an order of magnitude more complex than it needs to be. The net result is, after a few decades, you have hundreds or thousands of applications implemented, each with their own data model. You spend most of your IT budget on  systems integration, without really achieving  it. 

Semantic Arts discovered it’s possible to build an elegant core model of even the most complex enterprise in a limited amount of time (typically less than six months)  and with a limited amount of complexity  (typically fewer than 500 concepts (classes  + properties). When well implemented, the said model can be extended and specialized in handling specific requirements of subdomains or departments while still staying aligned with the enterprise core as business continues to grow. The model can be directly populated with data from legacy systems, and queries can be federated over the legacy systems through the core model. Such a model in the short term can provide a simple integration platform that not only integrates legacy systems but also allows the integration of unstructured and external data.

Meet the leader behind the success of  Semantic Arts. 

Dave McComb is the President and Co-founder of Semantic Arts. He and his team help organizations uncover the meaning in the data from their information systems. Dave is also the author of “The Data-Centric  Revolution”, “Software Wasteland”, and “Semantics in  Business Systems”. For 20 years, Semantic Arts has helped firms of all sizes in this endeavor, including Proctor &  Gamble, Goldman Sachs, Schneider-Electric, Lexis Nexis,  Dun & Bradstreet, and Morgan Stanley. Prior to Semantic  Arts, Dave co-founded Velocity Healthcare, where he developed and patented the first fully model-driven architecture. Prior to that, he was a part of the problem. 

“Semantic Arts exists to help organizations transition to a newly emerging paradigm of information systems based on flexible data structures and deep semantics. “

Semantic Arts is a professional services firm—they do not sell software or hardware—therefore, the company is completely objective and can bring business outcome thinking and considerable experience to every engagement. 

In conversation with  Dave McComb, Co-founder and CEO of Semantic Arts 

Can you tell us about your  services in brief? 

Semantic Arts delivers a single offering: we help guide clients who are committed to transitioning to a data-centric future by implementing this capability through our consulting services. Some may come to the need to do this through their digital transformation efforts, which more and more are discovering to be far harder than it should be because they are trying to build on a foundation of brittle complexity inherently in relational and legacy data structures. Some come to it through big data, AI, and  ML, when they find that they are spending most of their time and effort wrangling data that should already be organized. And some have come to it from first principles,  as we did. Our first engagement is a combination of what we call the “think big / start small” implementation methodology for most clients. We discovered the need to do both in parallel. To do only the think big  part (as we did for more than a  decade) creates beautiful models  that do not get implemented. To only start small (as every agile project does these days) almost guarantees the furtherance of silos.  By doing the two in parallel, we  can show how the core model (the  think big part) guides each “small” (incremental) project to conform to  the core. 

After that first project, most clients re-engaged us for many more projects, which either tackle additional domain areas, begin building out the information architecture needed to take maximum advantage of this approach, or establish training and governance to make this the new normal. 

Huge amount of data is created every second from sources like online transactions, social media,  or customer data. Do you think the existing ways of managing data is enough for exponentially bigger data quantity? 

Well, it has to be managed in layers,  it’s obvious that we will have even more data, and the Internet of Things is going to swamp the existing data completely. But the traditional approach of just copying all of the data and moving it around  to a big data platform so that people  can do analytics is not going to scale to that. There’s more data than what the systems can process over. What will happen instead is there’s going to be a lot more edge computing where the big data, IoT,  and the clickstream analysis will stay. It’s going to be abstracted, and the essential differences are going to come into an analytic framework. 

With the situation today due to coronavirus, everything is going online. How are you helping your clients adjust to the new normal? 

One of the big things that people  are doing right now is called digital transformation, and I think that the coronavirus pandemic has only accelerated the process. But as a lot of authors have pointed out, digital transformation projects have about a 70 percent failure rate, and they are incredibly expensive. People are just trying to use brute force to change their existing procedures  into a fully digital-enabled, stymied  sheer scale and complexity. Therefore, our approach is to find an elegant and simple model that represents the core of your business and use it to fuel your digital transformation.  We’re guiding our clients in implementing this capability to ease information findability, accessibility, interoperability, and reusability. 

Normally, reliable and high-quality services come at an expensive price tag. How do you manage to keep your services affordable? 

We primarily work with very large firms and have small, agile teams,  so compared to a typical application implementation project, these are a fraction of. Our services are very affordable and barely noticeable, but in the longer term, the data-centric approach is going to move into the mid-market, where it will become a platform play. 

What is a Market?  

The term “market” is a very common term in the business industry. We talk about the  automotive market, the produce market, the disk drive market, etc. And yet, what do we really mean when we use that term? It is an instructive question because very often CRM  (Customer Relationship Management) systems or other sales analytic systems group customers and sales based on their “market”. However, if we don’t have a clear understanding of what a market is, we misrepresent and misgroup and therefore mislead ourselves as we study the implication of our advertising and promotional efforts. 

The Historical Market. 

Historically, markets were physical places: marketplaces. People went to the market in  order to conduct trade. In many communities there was a single market where all goods were bought and sold. However, as communities became larger and developed into cities, marketplaces began to specialize and there was a particular place to go to buy and sell fresh produce. There was another place to go to buy and sell spices; and yet another to buy and sell furniture. 

As time progressed and the economy became more and more differentiated and cities  grew larger and larger, marketplaces became even more specialized and more geographically localized. So, for instance, we have the diamond marketplace in Antwerp,  Belgium, and the screenplay marketplace in Hollywood. 

Why Did Physical Marketplaces Emerge and Dominate? 

The trend toward physical marketplaces was not necessarily inevitable. Buyers could seek out sellers at their own place of business and conduct business that way.  Conversely, sellers could seek out buyers in their own place of business. What led to the popularity of the marketplace were two factors. One was that the physical movement to  the marketplace was collectively more efficient for most of the participants. The second  reason was that the marketplace allowed easy selection and comparison between similar  offerings. Additionally, the cost of information about potential sources or demands for a  given product or service was not nearly as economical as it is today with computers and  the Internet. 

Why Did Marketplaces Specialize? 

Marketplaces specialized because the sheer volume of selection and choice eventually became overwhelming. Imagine in the current day if we went to a marketplace where every possible product and service was available for sell from every possible supplier.  The marketplace would be so vast that none could traverse it. It certainly makes one wonder how COMDEX was as successful as it was. A general-purpose electronics show  offers far more variety than most participants would deal with. So, as the marketplaces  became larger, they began to self-organize into specialization of offerings.

But Why Did They Choose the Specializations That They Did? 

Why, for instance, did a marketplace in antique furniture develop and not a marketplace of chairs? Each is a specialization. The chair marketplace could conceivably offer a  wide variety of chairs, an infinite variety of chairs, including antique chairs. It would allow buyers to select the widest possible range of chairs. But that is not what occurred.  What occurred was an antique furniture marketplace where buyers could buy antique  chairs, antique tables, antique hutches, etc. And in parallel, a marketplace for  contemporary furniture has emerged. We see this in virtually every market that we look  at. There is not really a car market or marketplace. There is a luxury car market, an SUV  market, a luxury SUV market, etc. 

What Makes a Market? 

In modern times, we only occasionally rely on a physical marketplace to define a market.  Buyers and sellers get together and conduct business outside of a physical or even virtual marketplace. 

However, we still have the useful concept of a “market”. As with many other categories,  we group together or lump historical activity into categories with the intent that if the  categories are well-chosen and most members of the category are representative  members, then predictions made about the category will be valid for many of the  members. 

So what forms the basis of the categories? Categories can be based on attributes of the offering. So, for instance, we could make up a category of blue cars. Using blue as a possible attribute of cars and we could group buyers and sellers into the blue or not blue market place. But unless “blueness” was a selection metric, this is not how the market  will form.  

No, markets form when a group of customers begin selecting products or services based on one or two characteristics of an offering. For hundreds of years there have been potato markets. Generally they competed on the basis of price and perhaps freshness.  But when McDonalds began buying potatoes based on length and straightness (they wanted long french fries to stand up in those little cardboard envelops) a market emerged to satisfy this demand.  

In “The Innovators Dilemma” and “The Innovators Solution” Clayton Christensen outlines case after case of how markets formed around one set of metrics, but the incumbent vendors oversupplied the metric they were competing on, and left room for a new entrant to compete on a different metric. Eventually a new market emerged to serve a segment of the prior market who were more interested in the new metric of competition.  

So for instance, the computer disk drive market at one time primarily competed on capacity and price per MB of capacity. For years each season brought bigger and bigger drives at lower and lower cost per MB. At one of the firms the engineers “invented” a smaller (8”) disk drive, but none of the existing customers were interested in a drive with less capacity and greater cost per MB for what it had.

The engineers defected and started a new company which had rough going for a while until the mini computer market emerged, which had slightly different selection criteria.  The capacity of the mainframe drives already exceeded what they needed. They were  

more interested in total cost of the unit, and the footprint. A second market emerged to supply these metrics to these new customers. Christiansen goes on to describe how in some cases (the disk drive being one of them) that the continued improved in the new market eventually crowded into and displaced the established players from the existing market.  

For the purpose of this paper, though, we are more interested in how new markets come to be. And this is it. The way we currently define markets based on the SIC codes and  the like is deeply flawed and does not produce categories with comparable attributes.  Until we begin to discover what really creates and defines the boundaries around markets our categorizations are not going to be very useful.  

Christiansen goes on to say in the Innovator’s Solution that failing to distinguish which  markets you make offers to are in the, what he calls, “sustaining” category and which are  in the disruptive category, will lead to marketing and financial ruin for the unwary. 

Conclusion

To wrap it up, we believe that a market is a categorization based on a tacit agreement between a group of buyers and sellers as to what are the important selection criteria or selection metrics for the offering. Markets so defined do not have static long-term  boundaries; they are constantly in flux. Marketing executives and professionals involved  with analytics and business information would do well to align their market segments  with this definition, as we believe that the group of customers or suppliers who share  common selection criteria will also share much more common behavior and reception to  specific promotional activities.

Gartner has put “Knowledge Graphs” at the peak of inflated expectations. If you are a Knowledge Graph software vendor, this might be good news. Companies will be buying knowledge graphs without knowing what they are. I’m reminded of an old cartoon of an executive dictating into a dictation machine: “…and in closing, in the future implementing a relational database will be essential to the competitive survival of all firms. Oh, and Miss Smith, can you find out what a relational database is?” I imagine this playing out now, substituting “knowledge graph” for “relational database” and by-passing the misogynistic secretarial pool.

If you’re in the software side of this ecosystem, put some champagne on ice, dust off your business plan, and get your VCs on speed dial. Happy times are imminent.

Those of you who have been following this column know that our recommendations for data-centric transformations strongly encourage semantic technology and model driven development implemented on a knowledge graph. As we’ve said elsewhere, it is possible to become data-centric without all three legs of this stool, but it’s much harder than it needs to be. We find our fate at least partially tethered to the knowledge graph marketplace. You might think we’d be thrilled by the news that is lifting our software brethren’s boats.

But we know how this movie / roller coaster ends. Once a concept scales this peak, opportunists come out of the woodwork. Consultants will be touting their “Knowledge Graph Solutions” application and vendors will repackage their Content Management System or ETL Pipeline product as a key step on the way Knowledge Graph nirvana. Anyone who can spell “Knowledge Graph” will have one to offer.

Some of you will avoid the siren’s song, but many will not. Projects will be launched with great fanfare. Budgets will be blown. What is predictable is that these projects will fail to deliver on their promises. Sponsors will be disappointed. Naysayers will trot out their “I told you so’s.” Gartner will announce Knowledge Graphs are in the Trough of Disillusionment. Opportunists will jump on the next band wagon.

Click here to continue reading on TDAN.com

If you’re interested in Knowledge Graphs, and would like to avoid the trough of disillusionment, contact me: [email protected]

The Enterprise Ontology  

At the time of this writing almost no enterprises in North America have a formal enterprise ontology. Yet we believe that within a few years this will become one of the foundational pieces to most information system work within major enterprises. In this paper, we will explain just what an enterprise ontology is, and more importantly, what you can expect to use it for and what you should be looking for, to  distinguish a good ontology from a merely adequate one. 

What is an ontology?  

An ontology is a “specification of a conceptualization.” This definition is a mouthful but bear with me, it’s actually pretty useful. In general  terms, an ontology is an organization of a body of knowledge or, at  least, an organization of a set of terms related to a body of knowledge.  However, unlike a glossary or dictionary, which takes terms and provides definitions for them, an ontology works in the other direction.  An ontology starts with a concept. We first have to find a concept that  is important to the enterprise; and having found the concept, we need to express it in as precise a manner as possible and in a manner that can be interpreted and used by other computer systems. One of the differences between a dictionary or a glossary and ontology is, as we know, dictionary definitions are not really processable by computer systems. But the other difference is that by starting with the concept and specifying it as rigorously as possible, we get definitive meaning that is largely independent of language or terminology. Then the  definition states that an ontology is a “specification of a  conceptualization.” That is what we just described. In addition, of course, we then attach terms to these concepts, because in order for us humans to use the ontology we need to associate the terms that we commonly use. 

Why is this useful to an enterprise?  

Enterprises process great amounts of information. Some of this information is structured in databases, some of it is unstructured in documents or semi structured in content management systems.  However, almost all of it is “local knowledge” in that its meaning is agreed within a relatively small, local context. Usually, that context is  an individual application, which may have been purchased or may  have been built in-house.

One of the most time- and money-consuming activities that enterprise  information professionals perform is to integrate information from  disparate applications. The reason this typically costs a lot of money  and takes a lot of time is not because the information is on different  platforms or in different formats – these are very easy to  accommodate. The expense is because of subtle, semantic differences  between the applications. In some cases, the differences are simple:  the same thing is given different names in different systems. However,  in many cases, the differences are much more subtle. The customer in  one system may have an 80 or 90% overlap with the definition of a  customer in another system, but it’s the 10 or 20% where the  definition is not the same that causes most of the confusion; and there  are many, many terms that are far harder to reconcile than  “customer.” 

So the intent of the enterprise ontology is to provide a “lingua franca”  to allow, initially, all the systems within an enterprise to talk to each  other and, eventually, for the enterprise to talk to its trading partners  and the rest of the world. 

Isn’t this just a corporate data dictionary or consortia of data  standards?  

The enterprise ontology does have many similarities in scope to both a corporate data dictionary and consortia data standard. The similarity is primarily in the scope of the effort: both of those initiatives, as well as  

enterprise ontologies, aim to define the shared terms that an enterprise uses. The difference is in the approach and the tools. With both a corporate data dictionary and a consortia data standard the interpretation and use of the definitions is strictly by humans, primarily system designers. Within an enterprise ontology, the expression of the  ontology is such that tools are able to interpret and make inferences  on the information when the system is running. 

How to build an enterprise ontology  

The task of building an enterprise ontology is relatively straightforward. You would be greatly aided by purchasing a good  ontology editor, although reasonable ontology editors are available for  free. The analytical work is similar to building a conceptual enterprise data model and involves many of the same skills: the ability to form good abstractions, to elicit information from users through interviews,  as well as to find informational clues through existing documentation and data. One of the interesting differences is that as the ontology is being built it can be used in connection with data profiling to see whether the information that is currently being stored in information systems does in fact comply with the rules that the ontology would suggest. 

What to look for in an enterprise ontology  

What distinguishes a good or great enterprise ontology from a merely  adequate one are several characteristics that will mostly be exercised  later in the lifecycle of the actual use of the ontology. Of course, they  are important to consider at the time you’re building the ontology. 

Expressiveness 

The ontology needs to be expressive enough to describe all the distinctions that an enterprise makes. Most enterprises of any size at  all have tens of thousands to hundreds of thousands of distinctions  that they use in their information systems. Not only is each piece of schemata in all of their databases a distinction but so are many of the codes they have in code tables as well as decisions that are called out either in code or in procedure manuals. The sum total of all these distinctions is the operating ontology of the enterprise. However, they are not formally expressed in one place. The structure as well as the base concepts used need to be rich enough that when a new concept is uncovered it can be expressed in the ontology. 

Elegance 

At the same time, we need to strive for an elegant representation. It  would be simple but perhaps simplistic to take all the distinctions in all  the current systems and put them in a simple repository and call them  an ontology. This misses some of the great strengths of an ontology.  We want to use our ontology not only to document and describe  distinctions but also to find similarities. In these days of Sarbanes Oxley regulations it would be incredibly helpful to know which  distinctions and which parts of which schemas deal with financial  commitments and “material transactions.” 

Inclusion and exclusion criteria 

Essentially, the ontology is describing distinctions amongst “types.” In  many cases, what we would like to know is whether a given instance is  of a particular type. Let’s say it’s a record in a product table, therefore  it’s a type “product.” But in another system we may have inventory and we would like to know whether this instance is also compatible with the type that we’ve defined as inventory. In order to do this, we need in the ontology a way to describe inclusion and exclusion criteria:  what other clues we would use if we or another system were evaluating a particular instance to determine whether it was, in fact, of  a particular type. For instance, if inventory were defined as being physical goods held for resale, one inclusion criteria might be weight because weight is an indicator of a physical good. Clearly, there would be many more, as well as criteria for excluding. But this gives you an idea. 

Cross referencing capability 

Another criterion that is very important is the ability to keep track of where the distinction was found; that is, which system currently implements and uses this particular distinction. This is very important for producing any type of where-used information because as we change our distinctions it might have side effects on other systems. 

Inferencing 

Inferencing is the ability to find or infer additional information based on the information we have. For instance, if we know that an entity is a person we can infer that the person has a birthday, whether we know it or not, and we can also infer that the person is less than 150  years old. While this sounds simple at this level, the power in an ontology is when the inference chains become long and complex and we can use the inferencing engine itself to make many of these conclusions on-the-fly. 

Foreign-language support 

As we described earlier, the ontology is a specification of a conceptualization that we attach terms to. It doesn’t take much to add  the ability to add foreign language terms.. This adds a great deal of power for developers who wish to present the same information, and the same screens, in multiple languages, as we are really just manipulating the concepts and attaching the appropriate language at runtime. 

Some of these characteristics are aided by the existence of tools or  infrastructures, but many of them are produced by the skill of the  ontologist.

Summary  

We believe that the enterprise ontology will become a cornerstone in many information systems in the future. It will become a primary part of the systems integration infrastructure as one application will be translated into the ontology and we will very rapidly know what the corresponding schema and terms are and what transformations are needed to get to another application. It will become part of the corporate search strategy as search moves beyond mere keywords into actually searching for meaning. It will become part of business intelligence and data warehousing systems as naïve users can be led to similar terms in the warehouse repository and aid their manual search and query construction. 

Many more tools and infrastructures will become available over the  next few years that will make use of the ontology, but the prudent  information manager will not wait. He or she will recognize that there  is a fair lead time to learn and implement something like this, and any  implementation will be better than none because this particular  technology promises to greatly leverage all the rest of the system  technologies.

For the last 20 years,  Semantic Arts has been helping firms design and build enterprise ontologies to get them on the data-centric path. We have learned many lessons from the enterprise that can be applied in the construction of smart city ontologies.

What is similar between the enterprise and smart cities?

• They both have thousands of application systems. This leads to thousands of arbitrarily different data models, which leads to silos.
• The enterprise and smart cities want to do agile, BUT agile is a more rapid way to create more silos.

What is different between the enterprise and smart cities?

• In the enterprise, everyone is on the same team working towards the same objectives.
• In smart cities there are thousands of different teams working towards different objectives. For example:

Utility companies.
Sanitation companies.
Private and public industry.

• Large enterprises have data lakes in data warehouses.
• In smart cities there are little bits of data here and there.

What have we learned?

“Simplicity is the ultimate sophistication.”

20 years of Enterprise Ontology construction and implementation has taught us some lessons that apply to Smart City ontologies. The Smart City Landscape informs us how to apply those lessons, which are:

Think big and start small.

Simplicity is key to integration.

Low code / No code is key for citizen developers.

Semantic Arts gave this talk at the W3c Workshop on Smart Cities originally recorded on June 25, 2021. Click here to view the talks and interactive sessions recorded for this virtual event. A big thank you to W3C for allowing us to contribute!