Semantics and Ontology

What is Semantics?

Semantics is the study of meaning. By creating a common understanding of the meaning of things, semantics helps us better understand each other. Common meaning helps people understand each other despite different experiences or points of view. Common meaning in semantic technology helps computer systems more accurately interpret what people mean. Common meaning enables disparate IT systems – data sources and applications – to interface more efficiently and productively.

What is an Ontology?

An ontology defines all of the elements involved in a business ecosystem and organizes them by their relationship to each other. The benefits of building an ontology are:

• Everyone agrees on a common set of terms used to describe things
• Different systems – databases and applications – can communicate with each other without having to directly connect to each other.

Enterprise Ontology

An Ontology is a set of formal concept definitions.

An Enterprise Ontology is an Ontology of the key concepts that organize and structure an Organization’s information systems. Having an Enterprise Ontology provides a unifying whole that makes system integration bearable.

An Enterprise Ontology is like a data dictionary or a controlled vocabulary, however it is different in a couple of key regards. A data dictionary, or a controlled vocabulary, or even a taxonomy, relies on humans to read the definitions and place items into the right categories. An ontology is a series of rules about class (concept) membership that uses relationships to set up the inclusion criteria. This has several benefits, one of the main ones being that a system (an inference engine) can assign individuals to classes consistently and automatically.

By building the ontology in application neutral terminology it can fill the role of “common denominator” between the many existing and potential data sources you have within your enterprise. Best practice in ontology building favors building an Enterprise Ontology with the fewest concepts needed to promote interoperability, and this in turns allows it to fill the role of “least common denominator”

Building an Enterprise Ontology is the jumping off point for a number of Semantic Technology initiatives. We’ll only mention in passing here the variety of those initiatives (we invite you to poke around our web site to find out more) . We believe that Semantic Technology will change the way we implement systems in three major areas:

• Harvest – Most of the information used to run most large organizations comes from their “applications” (their ERP or EHR or Case Management or whatever internal application). Getting new information is a matter of building screens in these applications and (usually) paying your employees to enter data, such that you can later extract it for other purposes. Semantic Technology introduces approaches to harvest data not only from internal apps, but from Social Media, unstructured data and the vast and growing sets of publicly available data waiting to be integrated.
• Organize – Relational, and even Object Oriented, technology, impose a rigid, pre-defined structure and set of constraints on what data can be stored and how it is organized. Semantic Technology replaces this with a flexible data structure that can be changed without converting the underlying data. It is so flexible that not all the users of a data set need to share the same schema (they need to share some part of the schema, otherwise there is no basis for sharing, but they don’t need to be in lockstep, each can extend the model independently). Further the semantic approach promotes the idea that the information is at least partially “self-organizing.” Using URIs (Web based Uniform Resource Identifiers) and graph-based databases allows these systems to infer new information from existing information and then use that new information in the dynamic assembly of data structures.
• Consume — Finally we think semantic technology is going to change the way we consume information. It is already changing the nature of work flow-oriented systems (ask us about BeInformed). It is changing data analytics. It is the third “V” in Big Data (“Variety”). Semantic Based mashups are changing the nature of presentation. Semantic based Search Engine Optimization (SEO) is changing internal and external search.

Given all that, how does one get started?

Well you can do it yourself. We’ve been working in this space for more than twenty years and have been observing clients take on a DIY approach, and while there have been some successes, in general we see people recapitulating many of the twists and turns that we have worked through over the last decade.

You can engage some of our competitors (contact us and we’d be happy to give you a list). But, let us warn you ahead of time: most of our competitors are selling products, and as such their “solutions” are going to favor the scope of the problem that their tools address. Nothing wrong with that, but you should know going in, that this is a likely bias. And, in our opinion, our competitors are just not as good at this as we are. Now it may come to pass that you need to go with one of our competitors (we are a relatively small shop and we can’t always handle all the requests we get) and if so, we wish you all the best…

If you do decide that you’d like to engage us, we’d suggest a good place to get started would be with an Enterprise Ontology. If you’d like to get an idea, for your budgeting purposes, what this might entail, click here to get in touch, and you’ll go through a process where we help you clarify a scope such that we can estimate from it. Don’t worry about being descended on by some over eager sales types, we recognize that these things have their own timetables and we will be answering questions and helping you decide what to do next. We recognize that these days “selling” is far less effective than helping clients do their own research and supporting your buying process.

That said, there are three pretty predictable next steps:

• Ask us to outline what it would cost to build an Enterprise Ontology for your organization (you’d be surprised it is far less than the effort to build and Enterprise Data Model or equivalent)
• gist – as a byproduct of our work with many Enterprise Ontologies over the last decade we have built and made publicly available “gist” which is an upper ontology for business systems. We use it in all our work and we have made it publicly available via a Creative Commons Share Alike license (you can use it for any purpose provided you acknowledge where you got it)
• Training – if you’d like to learn more about the language and technology behind this (either through public courses or in house) check out of offerings in training.

How is Semantic Technology different from Artificial Intelligence?

Artificial Intelligence (AI) is a 50+ year old academic discipline that provided many technologies that are now in commercial use. Two things comprise the core of semantic technology. The first stems from AI research in knowledge representation and reasoning done in the 70s and 80s and includes ontology representation languages such as OWL and inference engines like Fact++. The second relates to data representation and querying using triple stores, RDF and SPARQL, which are largely unrelated to AI. A broad definition of semantic technology includes a variety of other technologies that emerged from AI. These include machine learning, natural language processing, intelligent agents and to a lesser extent speech recognition and planning. Areas of AI not usually associated with semantic technology include creativity, vision and robotics.

How Does Semantics Use Inference to Build Knowledge?

Semantics organizes data into well-defined categories with clearly defined relationships. Classifying information in this way enables humans and machines to read, understand and infer knowledge based on its classification. For example, if we see a red breasted bird outside our window in April, our general knowledge leads us to identify it as a robin. Once it is properly categorized, we can infer a lot more information about the robin then just its name.

We know for example that it is a bird; it flies; it sings a song; it spends its winter somewhere else and the fact that it has showed up means that good weather is on its way.

We know this other information because the robin has been correctly identified within the schematic of our general knowledge about birds, a higher classification; seasons, a related classification, etc.

This is a simple example of by correctly classifying information into a predefined structure we can infer new knowledge. In a semantic model, once the relationships are set up, a computer can classify data appropriately, analyze it based on the predetermined relationships and then infer new knowledge based on this analysis.

What is Semantic Agreement?

The primary challenge in building an ontology is getting people to agree about what they really mean when they describe the concepts that define their business. Gaining semantic agreement is the process of helping people understand exactly what they mean when they express themselves.

Semantic technologists accomplish this because they define terms and relationships independent from the context of how they are applied or the IT systems that store the information, so they can build pure and consistent definitions across disciplines.

Why is Semantic Agreement Important?

Semantic agreement is important because it is enables disparate computer systems to communicate directly with each other. If one application defines a customer as someone who has placed an order and another application defines the customer as someone who might place an order, then the two applications cannot pass information back and forth because they are talking about two different people. In a traditional IT approach, the only way the two applications will be able to pass information back and forth is through a systems integration patch. Building these patches costs time and money because it requires the owners of the two systems need to negotiate a common meaning and write incremental code to ensure that the information is passed back and forth correctly. In a semantic enabled IT environment, all the concepts that mean the same thing are defined by a common meaning, so the different applications are able to communicate with each other without having to write systems integration code.

What is the Difference Between a Taxonomy and Ontology?

A taxonomy is a set of definitions that are organized by a hierarchy that starts at the most general description of something and gets more defined and specific as you go down the hierarchy of terms. For example, a red-tailed hawk could be represented in a common language taxonomy as follows:

• Bird
  • Raptors
  • Hawks
    • Red Tailed Hawk

An ontology describes a concept both by its position in a hierarchy of common factors like the above description of the red-tailed hawk but also by its relationships to other concepts. For example, the red-tailed hawk would also be associated with the concept of predators or animals that live in trees.

The richness of the relationships described in an ontology is what makes it a powerful tool for modeling complex business ecosystems.

What is the Difference Between a Logical Data Model and Ontology?

The purpose of an ontology is to model the business. It is independent from the computer systems, e.g. legacy or future applications and databases. Its purpose is to use formal logic and common terms to describe the business, in a way that both humans and machines can understand. Ontologies use OWL axioms to describe classes and properties that are shared across multiple lines of business so concepts can be defined by their relationships, making them extensible to increasing levels of detail as required. Good ontologies are ‘fractal’ in nature, meaning that the common abstractions create an organizing structure that easily expands to accommodate the complex information management requirements of the business. The purpose of a logical model is to describe the structure of the data required for a particular application or service. Typically, a logical model shows all the entities, relationships and attributes required for a proposed application. It only includes data relevant to the particular application in question. Ideally logical models are derived from the ontology which ensures consistent meaning and naming across future information systems.

How can an Ontology Link Computer Systems Together?

Since an ontology is separate from any IT structure, it is not limited by the constraints required by specific software or hardware. The ontology exists as a common reference point for any IT system to access. Thanks to this independence, it can serve as a common ground for different:

• database structures, such as relational and hierarchical,
• applications, such as an SAP ERP system and a cloud-hosted e-market,
• devices, such as an iPad or cell phone.

The benefit of the semantic approach is that you can link the legacy IT systems that are the backbone of most business to exciting new IT solutions, like cloud computing and mobile delivery.

What are 5 Business Benefits of Semantic Technology Solutions?

Semantic technology helps us:

1. Find more relevant and useful information
   • Because it enables us to search information from disparate sources (federated search) and automatically refine our searches (faceted search).
2. Better understand what is happening
   • Because it enables us to use the relationships between concepts to predict and interpret change.
3. Build more transparent systems and communications
   • Because it is based on common meanings and mutual understanding of the key concepts and relationships that govern our business ecosystems.
4. Increase our effectiveness, efficiency and strategic advantage
   • Because it enables us to make changes to our information systems more quickly and easily.
5. Become more perceptive, intelligent and collaborative
   • Because it enables us to ask questions we couldn’t ask before.

How Can Semantic Technology Enable Dynamic Workflow?

Semantic-driven dynamic workflow systems are a new way to organize, document and support knowledge management. They include two key things:

1. A consistent, comprehensive and rigorous definition of an ecosystem that defines all its elements and the relationships between elements. It is like a map.
2. A set of tools that use this model to:
   • Gather and deliver ad hoc, relevant data.
   • Generate a list of actions – tasks, decisions, communications, etc. – based on the current situation.
   • Facilitate and document interactions in the ecosystem.

These tools work like a GPS system that uses the map to adjust its recommendations based on human interactions This new approach to workflow management enables organizations to respond faster, make better decisions and increase productivity.

Why Do Organizations Need Semantic-Driven, Dynamic Workflow Systems?

A business ecosystem is a series of interconnected systems that is constantly changing. People need flexible, accurate and timely information and tools to positively impact their ecosystems. Then they need to see how their actions impact the systems’ energy and flow. Semantic-driven, dynamic workflow systems enable users to access information from non-integrated sources, set up rules to monitor this information and initiate workflow procedures when the dynamics of the relationship between two concepts change. It also supports the definition or roles and responsibilities to ensure that this automated process is managed appropriately and securely. Organizational benefits to implementing semantic-driven, dynamic workflow systems include:

• Improved management of complexity
• Better access to accurate and timely information
• Improved insight and decision making
• Proactive management of risk and opportunity
• Increased organizational responsiveness to change
• Better understanding of the interlocking systems that influence the health of the business ecosystem

Blog post by Dave McComb

Click here to read a free chapter of Dave McComb’s book, “A Data-Centric Revolution”

Data-Centric vs. Software Wasteland

Dave McComb’s new book “Software Wasteland: How the Application-Centric Mindset is Hobbling our Enterprises” has just been released.

In it, I make this case that the opposite of Data-Centric is Application Centric, and our preoccupation with Application-Centric approaches over the last several decades has caused the cost and complexity of our information systems to be at least 10 times what they should be and in most cases we’ve examined, 100 times what they should be.

This article is a summary about how diametrically opposed these two world views are, and how the application-centric mind set is draining our corporate coffers.

An information system is essentially data and behavior.

On the surface, you wouldn’t think it would make much difference with which one you started with if you need both and they feed off each other.  But it turns out it does make a difference.  A very substantial difference.

What does it do?

The application-centric approach starts with “what does this system need to do?” Often this is framed in terms of business process and/or work flow.  In the days before automation, information systems were work flow systems.  Humans executed tasks or procedures.  Most tasks had prerequisite data input and generated data output.  The classic “input / process / output” mantra described how work was organized.

Information in the pre-computer era was centered around “forms.”  Forms were a way to gather some prerequisite data, which could then be processed.  Sometimes the processing was calculation.  The form might be hours spent and pay rate, and the calculation might be determining gross pay.

These forms also often were filed, and the process might be to retrieve the corresponding form, in the corresponding (paper) file folder and augment it as needed.

While this sounds like ancient history, it persists.  If you’ve been the doctor recently, you might have noticed that despite decades of “Electronic Medical Records,” the intake is weirdly like it always has been: paper form based.

This idea that information systems are the automation of manual work flow tasks continues.  In the Financial Service industry, it is called RPA (Robotic Process Automation) despite the fact that there are no robots.  What is being automated are the myriad of tasks that have evolved to keep a Financial Services firm going.

When we automate a task in this way, we buy into a couple of interesting ideas, without necessarily noticing that we have done so.  The first is that automating the task is the main thing.  The second is that the task defines how it would like to see the input and how it will organize the output.  This is why there are so many forms in companies and especially in the government.

The process automation essentially exports the problem of getting the input assembled and organized into the form the process wants.  In far too many cases this falls on the user of the system to input the data, yet again, despite the fact that you know you have told this firm this information dozens of times before.

In the cases where the automation does not rely on a human to recreate the input, something almost as bad is occurring: developers are doing “systems integration” to get the data from wherever it is to the input structures and then aligning the names, codes and categories to satisfy the input requirements.

Most large firms have thousands of these processes.  They have implemented thousands of application systems, each of which automates anywhere between a handful and dozens of these processes.  The “modern” equivalent of the form is the document data structure.  A document data structure is not a document in the same way that Microsoft Word creates a document. Instead, a document data structure is a particular way to organize a semi-structured data structure.  The most popular now is json (javascript object notation).

A typical json document looks like:

{‘Patient’: {‘id’: ‘12345’, ‘meds’: [ ‘2345’, ‘3344’, ‘9876’] } }

Json relies on two primary structures: lists and dictionaries.  Lists are shown inside square brackets (the list following the work ‘meds’ in the above example).  Dictionaries are key / value pairs and are inside the curly brackets.  In the above ‘id’ is a key and ‘12345’ is the value, ‘meds’ is a key and the list is the value, and ‘Patient’ is a key and the complex structure (a dictionary that contains a both simple values and lists) is the value.  These can be arbitrarily nested.

Squint very closely and you will see the document data structure is our current embodiment of the form.

The important parallels are:

• The process created the data structure to be convenient to what the process needed to do.
• There is no mechanism here for coordinating or normalizing these keys and values.

Process-centric is very focused on what something does.  It is all about function.

Click here to continue reading on TDAN.com

Complexity Drives Cost in Information Systems

A system with twice the number of lines of code will typically cost more than twice as much to build and maintain.

There is no economy of scale in enterprise applications.  There is dis economy of scale.   In manufacturing, every doubling of output results in predictable reduction in the cost per unit.  This is often called a learning curve or an experience curve.

Just the opposite happens with enterprise applications.  Every doubling of code size means that additional code is added at ever lower productivity.  This is because of complex dependency.  When you manufacture widgets, each widget has no relationship to or dependency on, any of the other widgets.  With code, it is just the opposite.  Each line must fit in with all those that preceded it.  We can reduce the dependency, with discipline, but we cannot eliminate it.

If you are interested in reducing the cost of building, maintaining, and integrating systems, you need to tackle the complexity issue head on.

The first stopping point on this journey is recognizing the role that schema has in the proliferation of code.  Study software estimating methodologies, such as function point analysis, and you will quickly see the central role that schema size has on code bloat.  Function point analysis estimates effort based on inputs such as the number of fields on a form, the elements in a transaction, or the columns in a report.  Each of these is directly driven by the size of the schema.  If you add attributes to your schema they must show up in forms, transactions, and reports, otherwise, what was the point?

I recently did a bit of forensics on a popular and well known high quality application: Quick Books, which I think is representative.  The Quick Books code base is 10 million lines of code.  The schema consists of 150 tables and 7500 attributes (or 7650 schema concepts in total).  That means that each schema concept, on average, contributed another 1300 lines of code to the solutions.  Given that most studies have placed the cost to build and deploy software at between $10 and $100 per line of code (it is an admittedly large range but you have to start somewhere) that means that each attribute added to the schema is committing the enterprise to somewhere between $13K and $130K of expense just to deploy, and probably an equal amount over the life of the product for maintenance.

I’m hoping this would give data modelers a bit of pause.  It is so easy to add another column, let alone another table to a design; it is sobering to consider the economic impact.

But that’s not what this article is about.  This article is about the insidious multiplier effect that not following the data centric approach is having on enterprises these days.

Let us summarize what is happening in enterprise applications:

• The size of each application’s schema is driving the cost of building, implementing, and maintaining it (even if the application is purchased).
• The number of applications drives the cost of systems integration (which is now 30-60% of all IT costs).
• The overlap, without alignment, is the main driver of integration costs (if the fields are identical from application to application, integration is easy; if the applications have no overlap, integration is unnecessary).

We now know that most applications can be reduced in complexity by a factor of 10-100.  That is pretty good.  But the systems of systems potential is even greater.  We now know that even very complex enterprises have a core model that has just a few hundred concepts.  Most of the rest of the distinctions can be made taxonomically and not involve programming changes.

When each sub domain directly extends the core model, instead of the complexity being multiplicative, it is only incrementally additive.

We worked with a manufacturing company whose core product management system had 700 tables and 7000 attributes (7700 concepts).  Our replacement system had 46 classes and 36 attributes (82 concepts) – almost a 100-fold reduction in complexity.  They acquired another company that had their own systems, completely and arbitrarily different, smaller and simpler at 60 tables and 1000 attributes or 1060 concepts total.  To accommodate the differences in the acquired company we had to add 2 concepts to the core model, or about 3%.

Normally, trying to integrate 7700 concepts with 1060 concepts would require a very complex systems integration project.  But once the problem is reduced to its essence, we realize that there is a 3% increment, which is easily managed.

What does this have to do with data centricity?

Until you embrace data centricity, you think that the 7700 concepts and the 1060 concepts are valid and necessary.  You’d be willing to spend considerable money to integrate them (it is worth mentioning that in this case the client we were working with had acquired the other company ten years ago and had not integrated their systems, mostly due to the “complexity” of doing so).

Once you embrace data centricity, you begin to see the incredible opportunities.

You don’t need data centricity to fix one application.  You merely need elegance.  That is a discipline that helps guide you to the simplest design that solves the problem.  You may have thought you were doing that already.  What is interesting is that real creativity comes with constraints.  And when you constrain your design choice to be in alignment with a firms’ “core model,” it is surprising how rapidly the complexity drops.  More importantly for the long-term economics, the divergence for the overlapped bits drops even faster.

When you step back and look at the economics though, there is a bigger story:

The total cost of enterprise applications is roughly proportional to:

These items are multiplicative (except for the last which is a divisor).   This means if you drop any one of them in half the overall result drops in half.  If you drop two of them in half the result drops by a factor of four, and if you drop all of them in half the result is an eight-fold reduction in cost.

Dropping any of these in half is not that hard.  If you drop them all by a factor of ten (very do-able) the result is a 1000 fold reduction in cost.  Sounds too incredible to believe, but let’s take a closer look at what it would take to reduce each in half or by a factor of ten.

Click here to read more on TDAN.com

We have taken the position that a core model is an essential part of your data-centric architecture. In this article, we will review what a core model is, how to go about building one, and how to apply it both to analytics as well as new application development.

What is a Core Model?

A core model is an elegant, high fidelity, computable, conceptual, and physical data model for your enterprise.

Let’s break that down a bit.

Elegant

By elegant we mean appropriately simple, but not so simple as to impair usefulness. All enterprise applications have data models. Many of them are documented and up to date. Data models come with packaged software, and often these models are either intentionally or unintentionally hidden from the data consumer. Even hidden, their presence is felt through the myriad of screens and reports they create. These models are the antithesis of elegant. We routinely see data models meant to solve simple problems with thousands of tables and tens of thousands of columns. Most large enterprises have hundreds to thousands of these data models, and are therefore attempting to manage their datascape with over a million bits of metadata.

No one can understand or apply one million distinctions. There are limits to our cognitive functioning. Most of us have vocabularies in the range of 40,000-60,000, which should suggest the upper limit to a domain that people are willing to spend years to master.

Our experience tells us that at the heart of most large enterprises lays a core model that consists of fewer than 500 concepts, qualified by a few thousand taxonomic modifiers. When we use the term “concept” we mean a class (e.g., set, entity, table, etc.) or property (e.g., attribute, column, element, etc.). An elegant core model is typically 10 times simpler than the application it’s modeling, 100 times simpler than a sub-domain of an enterprise, and at least 1000 times simpler than the datascape of a firm.

Click here to continue reading on TDAN.com

There is a movement afoot. I’m seeing it all around me. Let me outline some of the early outposts.

Data-Centric Manifesto

We put out the data-centric manifesto on datacentricmanifesto.org over two years ago now. I continue to be impressed with the depth of thought that the signers have put into their comments. When you read the signatory page (and I encourage you to do so now) I think you’ll be struck. A few just randomly selected give you the flavor:

This is the single most critical change that enterprise architects can advocate – it will dwarf the level of transformation seen from the creation of the Internet. – Susan Bright, Johnson & Johnson

Integration Debt is a Form of Technical Debt

As with so many things, we owe the coining of the metaphor “Technical Debt” to Ward Cunningham and the agile community. It is the confluence of several interesting conclusions the community has come to. The first was that being agile means being able to make a simple change to a system in a limited amount of time, and being able to test it easily. That sounds like a goal anyone could get behind, and yet, this is nearly impossible in a legacy environment. Agile proponents know that any well-intentioned agile system is only six months’ worth of entropy away from devolving into that same sad state where small changes take big effort.

One of the tenants of agile is that patterns of code architecture exist that are conducive to making changes. While these patterns are known in general (there is a whole pattern languages movement to keep refining the knowledge and use of these patterns), how they will play out on any given project is emergent. Once you have a starting structure for a system, a given change often perturbs that structure. Usually not a lot. But changes add up, and over time, can greatly impede progress.

One school of thought is to be continually refactoring your code, such that, at all times, it is in its optimal structure to receive new changes. The more pragmatic approach favored by many is that for any given sprint or set of sprints, it is preferable to just accept the fact that the changes are making things architecturally worse; as a result, you set aside a specific sprint every 2-5 sprints to address the accumulated “technical debt” that these un-refactored changes have added to the system. Like financial debt, technical debt accrues compounding interest, and if you let it grow, it gets worse—eventually, exponentially worse, as debt accrues upon debt.

Integration Debt

I’d like to coin a new term: “integration debt.” In some ways it is a type of technical debt, but as we will see here, it is broader, more pervasive, and probably more costly.

Integration debt occurs when we take on a new project that, by its existence, is likely to lead someone at some later point to incur additional work to integrate it with the rest of the enterprise. While technical debt tends to occur within a project or application, integration debt takes place across projects or applications. While technical debt creeps in one change at a time, integration debt tends to come in large leaps.

Here’s how it works: let’s say you’ve been tasked with creating a system to track the effectiveness of direct mail campaigns. It’s pretty simple – you implement these campaigns as some form of project and their results as some form of outcomes. As the system becomes more successful, you add in more information on the total cost of the campaign, perhaps more granular success criteria. Maybe you want to know which prospects and clients were touched by each campaign.

Gradually, it dawns that in order to get this additional information (and especially in order to get it without incurring more research time and re-entry of data), it will require integration with other systems within the firm: the accounting system to get the true costs, the customer service systems to get customer contact information, the marketing systems to get the overlapping target groups, etc. At this point, you recognize that the firm is going to consume a great deal of resources to get a complete data picture. Yet, this could have been known and dealt with at project launch time. It even could have been prevented.

Click here to read more on TDAN.com