Our Vision

Introduction

You certainly know the principles of relational databases, object databases or even XML databases. Although relational databases are the most used and also the cheapest ones, they still remain not as flexible and sometimes even not as fast as object or XML systems. Actually, each system has its pros and cons and too much concepts have to be taken into account to decide which one is best, it’s a choice that has to be taken with the whole information system in mind.

In this document we will discover a new way of managing data which should fill in the technical gap between all the different existing solutions, by using the Evaluant Universal Storage Service framework (EUSS).

EUSS unifies data programming across data source types by providing a new logical architecture for managing data on the .Net framework. Besides it ships with several technical services allowing creating from simple to very complex information systems, with the same ease of use.

Concepts

Entities, Relationships and Attributes

Description

ERA modelling was first introduced by Peter Chen in 1976 [Chen76]. This method allows representing any real world object, their relationships and their characteristics. Contrary to ADO.Net which gives a relational logical model for manipulating data, EUSS is based on the ERA model allowing straightforward access to an object graph compared to a relational model. Moreover, basing all the data on an ERA principle allows a better interoperability between disparate data store technical infrastructures like relational, object and XML databases. The ERA paradigm becomes a pivot format for any type of data in a very natural manner as you will be able to see all along this document.

By using ERA concepts, we can transform any piece of data in a set of three constituents.

Entity: It is the core element about the data to represent. Entities are in general recognizable, concrete or abstract, like a person, a place, a thing.

Relationship: Relationships allow creating a link between two entities. They are in a general manner typed so that they can associate two same entities with different relationships

Attribute: The attributes are associated to an entity to describe it.

Euss Core API

EUSS provides an implementation of the ERA model. Using this API you can manipulate graphs of entities, of any type, with any attributes. If you understand what is the DataSet for ADO.Net, then you may understand that there is the same relationship between an Entity and EUSS.

Here is a simple use of the model provided by EUSS. In this example we create an object graph composed by two entities linked to each other with the role named Partners, and each of them owning three attributes: Firstname, Lastname and Age.

// creates a first entity
Entity p1 = new Entity("Person");
p1.AddValue("Firstname", "Bill");
p1.AddValue("Lastname", "Gates");
p1.AddValue("Age", 13);

// and a second one
Entity p2 = new Entity("Person");
p2.AddValue("Firstname", "Steeve");
p2.AddValue("Lastname", "Balmer");
p2.AddValue("Age", 13);

// adds an association between the thwo
p1.AddValue("Partners", p2);

Serializing graphs

Now if we wanted to store those entities directly into a relational database, we would instinctively create two records in the same Person table. We could also put them into an XML document by creating two tags with the specific attributes. For EUSS this is all transparent as from a logical point of view these are only two entities. So we keep thinking with the same concepts without having to foresee what we will do with it. In EUSS this is only the matter of what is called a Persistence Engine which cotains all the logic about persisting a graph.

Here is a schema representing the main components provided by Euss.

Persistence Engines

Introduction

EUSS is not only an object model. It is also a complete persistence infrastructure allowing you to define complex information systems with ease, through what we call “Persistence Engines”. Those engines implement the IPersistenceEngine interface. Any class implementing this interface must be able to serialize an ERA object graph into a physical data store, as querying some data using an unified query language. For instance we could imagine an IPersistenceEngine implementation providing XML serialization, relational database management, or any other physical storage. Besides, those engines can provide specific functionalities as we will see.

IPersistenceEngine pe = new XmlPersistenceEngine("c:\repos.xml");

Transaction t = new Transaction();
t.Serialize(p1);
t.Commit(pe);

This example saves the object graph we have already created into an XML file. If we would have wanted to persist it into a relational database, we should have used another persistence engine like this one:

IPersistenceEngine pe = new SqlPersistenceEngine(connectionString);

By abstracting the serialization mecanism we can store any graph of entities in potentilay any data store, providing we have a specific persistence engine implementation. By the way there is also a mecanism to declaratively define persistence engines's configurations. This is the next logical step of abstraction, using a factory to instantiate our persistence engine. You can use the XmlConfigLoader for this:

IPersistenceEngine pe = XmlConfigLoader.LoadXmlConfig("engines.config");

And the corresponding configuration file would look like this:

<?xml version="1.0" encoding="utf-8" ?> 
<PersistenceEngines xmlns="http://euss.evaluant.com/schemas/EngineConfiguration.xsd" DefaultEngine="myConfig">
  <PersistenceEngine Name="myConfig" Factory="UniversalStorageServices.Xml.XmlFactory">
    <FileName>C:\uss.xml</FileName> 
  </PersistenceEngine>
</PersistenceEngines>

A configuration file can contain several distinct configurations, and you can define which one will be used by the calling the application.

Serializing anywhere

Euss ships with a set of fully functional persistence engines of many types. Some are specific for data storage, while others are architectural one providing further services like caching, remoting, tracing, replication and more. Among the storage engines you can find an XML serializer handling cyclic relationchips, a generic relational database engine, storing any data in a three-table RDBMS instance, an In Memory Database also called "Prevalence", and also an object-relational mapping engine if you need to use an existing schema or define one. All those engines implement the same services, and you can switch from one to another just by changing the engines configuration file described earlier.

Unified requests: OPath

Introduction

A data storage service must always provide a way to query the data. The most well known ones are currently SQL for relational databases and XPath for XML documents. As EUSS is made to allow a logical abstraction for storage services, it also has to provide an abstraction for querying mechanism, using the same language whatever the used IPersistenceEngine.

For this a unified language has been created. This is the responsibility of each persistence engine implementation to be able to query an object graph using this language, often by transforming a request into its own language. This language was named OPath, for “object path”.

Architecture

The core of EUSS provides a default implementation of an OPath parser that is able to give an object graph representation of any OPath request string.

Examples

This OPath parser is used by all persistence engines to generate an object graph from a request string representation. Then they transform this graph into specific representations for each physical data store. For instance, in the case of a relational database, this request will be translated into a SQL request, specific to the used database system. In the case of XML storage it will be in XPath.

IPersistenceEngine pe = XmlConfigLoader.LoadXmlConfig("engines.config");

EntitySet people = pe.Load("Person[Lastname = 'Gates']");

foreach(Entity e in people)
    Console.WriteLine(e["Firstname"]);

Based on the XPath behaviour, you can query and navigate object like you do with nodes. OPath is flexible enough to let you use aggregation functions, and set constraints among differents paths in your object model. You can also use pagination and ordering whatever the datastore is.

Serializing CLR objects

Mapping object to entities

In the EUSS world we can manipulate entities. An entity as a type some attributes and relationships with other entities. Objects have also the same characteristics which are inherited by the class they are the instance of. For this reason, instances of CLR classes can be seen as entities. There is a straight forward mapping between entities and objects. Thus we can found technical solutions to automatically create entities when we instantiate objects of specific classes.

There is a no ending discussion about the technique to use by standard object-relational mapping frameworks. This is about whether to use the code generation or some reflection/emit methods. In Euss not only did we decide to comply with both of them, letting the user choose the one that most suits to his needs, but we also implemented an exclusive AOP technique. Thus you have access to three totally different possibilities:

Code Generation in C#, VB.Net, with 1.1 or 2.0 code compatibility
Dynamic Proxies with minimal constraints on your domain (e.g. no inheritance, nor reference to Euss), allowing you to reuse existing domain models
Aspect Oriented Programming, with an exlusive technique allowing you to still debug you classes

Here is an example of how to persists instances of a simple Person class. We while suppose that any of the three techniques described above to create your class has been chosen.

// Loads a specific configuration and the corresponding Persistence Engine
ObjectService os = new ObjectService("engines.config");
ObjectContext oc = os.CreateObjectContext();

// Creates a simple object
Person p = new Person();
p.Firstname = "Bill";
p.Lastname = "Gates";

// Stores the object in a datastore
oc.BeginTransaction();
oc.Serialize(p);
oc.CommitTransaction();

Now here is the code to use to load this object:

// Set no contraint to the type Person, and returns a list of Person objects
IList<Person> people = oc.Load<Person>(typeof(Person));

foreach(Person item in people)
    Console.WriteLine("{0} {1}", p.Firstname, p.Lastname);

Creating Complex Systems

Start simply

When creating any application we always start by creating what is called a prototype to verify the concepts. At this step we may not want to take a lot of time focusing on the persistence layer, i.e. we just want things to get serialized.

Though the standard .Net XML or Binary serialization are not enough in the way they don't provide query functionnalities or event don't cope with some complex object graphs. Euss is the ideal way of starting an application without worrying about the data access layer, as you can use with exactly the same code either a simple local XML file or a complex distributed relational database with a cache layer.

At this point, you just focus on your application logic, not worrying on how it is serialized because you don't need performance. Later, when you will need a specific persistence mecanism, you will configure this system without changing any line of your code, and the modifications will be propagated to Euss.

Go far

Euss ships with several Persistence Engines. Some are infrastructure ones, allowing you to plug them serially or parallelly. For instance there is a Hub Engine which is able to clone serialization command and dispatch them to sub engines. The result is a replicated persistence system, for instance when you want to have a main datastore, and a backup one in case the first fails.

For instance you can also use this Hub together with the Trace Engine displaying all the executed commands to the Tace's output stream.

In a standard production environment you could use this sort of configuration:

In this exemple, a Cache Engine is shared among all client application, for instance Windows Forms clients. This is done by using a Remoting Engine on the client, telling the Euss layer to use a remoted engine. On the Data Server side, a service is running handling all the requests. Thoses requests go through the same Cache Engine, whoever the client is, preventing from standard Cache inconsistency. If you need greater performance than the cache can provide you can also switch this with a Memory Engine, representing a "everything in memory" cache. Finally, the cache (or the memory) engine will dispatch the commands to an Object-Relational Mapping engine, allowing you to use you preferred database system, would it be Microsoft SQL Server but also Microsoft Access, Oracle or MySQL.