Do You Want a Semantic Layer?

I co-founded my firm to give attention to the challenges of supporting numerous knowledge analysts engaged on disparate units of information managed in a large lake. We borrowed the time period “semantic layer” from the parents at Enterprise Objects, who initially coined it within the Nineteen Nineties. The time period was really over 20 years outdated once we adopted it. 

So what’s a semantic layer precisely? If you happen to Google the time period, the next definition will pop up, which is a reasonably darn good definition for my part:

“A semantic layer is a enterprise illustration of company knowledge that helps finish customers entry knowledge autonomously utilizing frequent enterprise phrases. A semantic layer maps complicated knowledge into acquainted enterprise phrases corresponding to product, buyer, or income to supply a unified, consolidated view of information throughout the group.”

Wikipedia defines a semantic layer as a enterprise illustration of information that permits finish customers to entry knowledge autonomously. Everybody can agree {that a} business-friendly view of information that gives customers with self-service entry to analytics is fascinating – true knowledge democratization. It’s straightforward to see why it’s elementary to scaling knowledge and analytics.

Key Indicators That You Want a Semantic Layer

So how are you aware that you simply want a semantic layer? On this article, we’ll ask some powerful questions that may assist you reply that query. If you happen to reply “sure” to the next questions, your group in all probability wants a semantic layer.

1. Will we use a couple of BI/AI software or knowledge platform?

The bigger the group, the more durable it turns into to impose a single commonplace for consuming and getting ready analytics. Not solely is trying to vary person’s habits typically futile, it creates a barrier to customers making data-driven selections as a result of they should study new methods of asking questions. In line with the Dresner’s Knowledge of Crowds® Enterprise Intelligence Examine, over half of enterprises report utilizing three or extra BI instruments, with over a 3rd utilizing 4 or extra. On high of BI customers, knowledge scientists have their very own vary of choice as do software builders.

Along with a fancy analytics consumption panorama, knowledge storage and serving might be much more complicated. Information can stay in on-premise knowledge warehouses, cloud knowledge warehouses, knowledge lakes, or SaaS functions, making it troublesome for customers to seek out, mix and question knowledge.

A semantic layer supplies a constant, business-friendly interface for any question software and hides how and the place knowledge is saved.

2. Do customers present an absence of belief in knowledge and analytical outcomes?

Most organizations don’t belief their knowledge, resulting in sluggish selections or no selections in any respect. In actual fact, in keeping with the latest Chief Information Officer Survey, 72% of information and analytics leaders are closely concerned in or main digital enterprise initiatives, however they’re unsure how they will construct a trusted knowledge basis to speed up them.

It’s not onerous to see why an absence of belief in analytics outputs is so pervasive. Conflicting analytics outputs are all however assured when a number of enterprise items, teams, enterprise customers, and knowledge scientists put together their analytics utilizing their very own enterprise definitions and their very own instruments.

A semantic layer can drive belief in knowledge by empowering knowledge self-service whereas guaranteeing the consistency, constancy, and explainability of analytic outputs.

3. Do customers complain that they will’t get entry to knowledge after they want it?

With the quick tempo of in the present day’s enterprise local weather, ready for a centralized knowledge group to provide analytics for the enterprise is a factor of the previous. The self-service analytics revolution was born in response to the necessity for companies to free themselves from the constraints of IT. What appeared like a hit at first, nonetheless, slowly turned a quagmire as a result of self-service compelled enterprise customers to turn out to be knowledge engineers.

In consequence, in the present day’s data-driven decision-making is restricted to the realm of the superior SQL jockeys, resulting in frustration for almost all of customers and shifting the bottleneck to knowledge engineers, as a substitute of IT.

A semantic layer accelerates knowledge entry by making business-friendly knowledge accessible to everybody, not simply knowledge engineers or SQL consultants.

4. Are we reluctant to share knowledge?

Information governance and safety should not binary. It’s not sufficient to simply prohibit entry fully. Quite, a helpful knowledge safety and governance resolution will be sure that knowledge is seen (both fully or masked) to customers and teams relying on their authorization degree. For instance, the finance group of a public firm with insider standing could have entry to income knowledge whereas the advertising group doesn’t, and the HR division could have entry to full social safety numbers of its workers whereas the remainder group can solely see the final 4 digits.

Implementing a complete safety and governance technique in your knowledge yields advantages far past simply securing knowledge entry. With the arrogance that your knowledge is persistently safe for each sort of entry, organizations could make all knowledge out there to their workers and companions. Nonetheless, reaching constant governance in a fancy surroundings with a number of entry vectors (i.e., BI instruments, AI/ML instruments, functions) and a number of knowledge shops (i.e., knowledge lakes, knowledge warehouses, SaaS functions) is unattainable with no single management aircraft to use knowledge safety and governance at question time. 

A semantic layer applies knowledge safety and governance to each question by imposing entry insurance policies and guidelines to customers and teams in actual time, making knowledge sharing ubiquitous.

5. Are customers resorting to creating knowledge extracts or imports to get the question efficiency they want? 

Customers demand knowledge entry on the pace of thought. Cloud knowledge platforms have improved question pace and scale dramatically, however they’re nonetheless not quick sufficient to ship queries below a second persistently. Ready 10, 20, 30 seconds, or longer for a question is just not acceptable and customers will discover a method to obtain the pace they want by resorting to knowledge copies or cubing options like Tableau Hyper Extracts and Energy BI Premium Imports. This resolution is suboptimal as a result of it creates knowledge copies and knowledge latency, and requires processes to replace these caches. Moreover, exterior caching schemes additionally introduce safety considerations and infrequently create inconsistency in outcomes provided that knowledge could also be captured at totally different time intervals. 

The choice to avoiding knowledge motion and exterior caches is to ship a stay, performant connection to knowledge the place question efficiency is adaptively tuned and queries are rewritten in actual time.

A semantic layer leverages the facility of cloud knowledge platforms by autonomously managing question efficiency in situ utilizing finish person question patterns and machine-generated aggregates to ship queries in below one second.

Abstract

As you possibly can see above, a semantic layer can take away friction and make knowledge out there to everybody in your group, not simply knowledge engineers or SQL jockeys. Not surprisingly, a common semantic layer is turning into a important part in a contemporary knowledge and analytics stack.