Raijin Database Engine

The Schemaless SQL Database built ​for today's data challenges


Most SQL systems were designed for transactional data processing. Many of these OLTP systems work with data one tuple at a time and are unsuitable for processing large amounts of data especially when analytical queries are involved. Analytic RDBMS products - mostly those that have a columnar architecture - generally do a much better job but are still plagued by the limitations of SQL when it comes to processing this kind of data. Rigid database schemas get in the way of application development and ongoing schema maintenance becomes a burden. Due to schema rigidity, data needs to be denormalized and stored in different tables. This introduces another complexity cost while performance also suffers.

To come around these limitations IT engineers frequently combine document databases or other NoSQL solutions with RDBMS products in order to store and process denormalized data. Unfortunately most of these document database systems suffer from the same issue that OLTP systems have when it comes to analytical type workloads. Some don't support any analytical queries at all and the sole purpose of using a NoSQL solution is to be able to easily store semi-structured data.

Support For 
Semi-Structured Data

One of the primary goals of Raijin is to natively support semi-structured data to provide the flexibility that made document databases popular.

A big limitation of traditional SQL systems is schema rigidity.  The Raijin database does not require a schema to be defined up-front allowing you to cope with data variety as records may contain different fields. The database does not store NULL values in order to efficiently deal with sparse data.

If you want schema enforcement, explicitly listing the column names and types in the CREATE TABLE statement will also work in order to remain compatible and ensure that your existing SQL code would still work. The difference from traditional SQL systems is that Raijin will accept records with field names that were not defined beforehand and it will allow using this data as if it were a regular column.

This allows denormalized data to be painlessly stored and processed.

Raijin Database - Support For  Semi-Structured Data
Raijin Database supports SQL as its primary query language while lifting some of SQL's limitations.

Using SQL

SQL is a great declarative query language that all data analysts are familiar with.

The Raijin Database supports SQL as its primary query language while lifting some of SQL's limitations. It tries to adhere to the PostgreSQL syntax and also supports some other dialects that we considered useful.

In addition to lifting the requirement of having to define a schema, Raijin will also allow operations on some types that are forbidden in standard SQL. For example when an operation is attempted on a string (i.e. varchar) type, a conversion is done to deduce the required type and proceed with the operation instead of giving an error. The side-effect of being schemaless allows the same field (i.e. column) to contain different types and this automatic type conversion behavior aims to help with the veracity of the data. This makes Raijin's SQL dialect similar to a weakly typed language.

JSON Data Representation

The JSON data format has quickly become the defacto standard for data interchange in recent years.

In order to deal with sparse data, the Raijin Database uses a flat JSON representation for the data records. This is natively supported when loading and querying data unlike in other popular SQL solutions where this got introduced as a bolt-on afterthought. The JSON format is used both in query output as well as when loading data. Note that only flat JSON (i.e. a list of key-value pairs) is supported, unfortunately Raijin can't deal with nested data structures such as arrays and maps. For compatibility with existing SQL code, the traditional CSV format is also supported when loading data.

The support for the JSON format coupled with the REST API allows developers to get data in and out of the database easily, especially when interfacing with web-based applications.

Raijin Database - JSON Data Representation
what's next

The Road Ahead

The Raijin Database engine is nowhere near complete. There is still an extreme amount of work to be done on features currently missing that we intend to implement, including but not limited to the following:

  • Joins,
  • Subselects,
  • Window functions, common table expressions,
  • Update, upsert, delete,
  • Views,
  • A great deal of optimizer wizardry,
  • Full-text search,
  • Scale-out distributed cluster mode,
  • Authentication and access control,
  • Language bindings,

On the other hand there are features found in other SQL solutions that are not in-line with the design goals of Raijin and will probably not be implemented in the near future:

  • Transactions (commit/rollback),
  • Foreign key constraints,
  • Triggers, stored procedures,
  • User defined functions and types,
  • Cursors.

Start using the Raijin database for free

The schemaless database engine built for today's data challenges