CppNoSqlDb Repository

Demonstration prototype for No SQL database

Quick Status

Code functions correctly

no known defects

Demonstration code

yes

Documentation

yes

Test cases

Static library

Build requires

C++17 option

Planned design changes

Add example application using NoSqlDb

1.0 Concept

Fig 1. NoSqlDb Packages

Fig 2. DbCore Code

Fig 3. DbElement Code

The data models used by noSQL databases are usually based on key/value pairs, document stores, or networks. noSQL processing favors modeling flexibility, the ability to easily scale out across multiple machines, and performance with very large datasets. For that flexibility they give up real-time data consistency, accepting application enforced eventual consistency. They give up a formal query mechanism (hence the name). And, they may give up Durability guarantees by only occasionally writing to persistant storage in order to provide high throughput with large volumes of data.

The choice to use SQL or noSQL data management is driven by the nature of its uses. Managing metadata in a large repository, for example, could be well suited for implementation with a NoSQL database.

2.0 Design

This prototype design is factored into several distinct parts:

DbCore<P> package: provides an in-memory key-value database, based on the C++ std::unordered_map<key, value>. It provides only in-memory storage, leaving other database operations, like querys and pesistance to other packages.
DbElement<P> package: represents a single record in the database. It contains metadata items name, time-date, and children. Children provide a way to easily establish relationships between records, perhaps based on dependencies. The template parameter P (Payload) represents the data we want to store, and the metadata simply provides information about each of the P values.
Query<P> package: supports extracting information from the database. Queries start with a key set, looking for values and or keys that satisfy some condition, and returns a new key set with all the keys for records that match. Note that this design supports making compound queries where each subquery operates on the key set returned by its predecessor subquery, resulting in a progressive narrowing of the returned keys. You might think of this Query process as returning a new, probably smaller database, e.g., set of keys and values.
Persist<P> package: provides durable storage for the database or some shard of the database. Persist simply makes a query, usually returning the entire database, and saves that to a file. In this design we use XML storage, but JSON data or some other data format would work as well, and it is easy to configure the database for alternate storage formats by using a different Persist package. Note that it is, with this design, easy to shard partial contents of the database to a shard file. The only thing required to shard is to make a query that returns less than the entire database.

Note that we can easily provide the equivalent of stored procedures by defining and saving lambdas, each of which execute some specific query. These could be saved in a container for future use. In fact, that container could be just another instance of NoSqlDb.

3.0 Implementation

NoSqlDb code was built with Visual Studio Community Edition - 2019 and tested on Windows 10.

4.0 Status

The code has a demonstration package, TestClass, that was used to demonstrate how a CSE687 course project could convincingly show that it met project requirements (Project #1, Spring 2018).

I intend to use this facility in some up-coming projects, but it has not yet been used in a larger application, so it is possible there are latent errors that have not yet be discovered.

This demonstration needs an example application that creates and uses NoSqlDb intance(s).