SQLFire the NewSQL Database- Part-I

Now in this cloud computing world importance is given to data rather than the Compute infrastructure.

Clearly there is a change in database need over the years.

So what is driving this change in database need  ?

Today’s applications are highly distributed and typically web centric and need to support multiple types of devices. They need to be integrated with social and collaborative applications provide real time support to these applications.

So developers are looking for new techniques and technology to support these new applications from data side.

They are looking NoSQL  as an alternative to traditional relational database management systems. They are developing with higher level abstractions and frameworks like the spring framework in Java. Developers are writing applications much more quickly than in the past days of very lengthy product release cycles spanning 18 months to 24 months. Now these cycles have been much more shorter and quick iterations ( 3 months cycle) putting pressure on existing data model.

Also the infrastructure world  evolved quite a lot, it has moved from Physical to virtualized and now into cloud. So all these things driving a lot of change in how do we handle data. Some of the emerging trend of having Low latency ,high performance, real time aspect applications. Those are driving some of the changes in the database layer. Global Distribution, the ability to have data distributed across geographies and regions, data is just no longer silos into a single location or single database platform, high scalability, BigData analytics.

These are some of the trends and requirement that is driving the NoSQL and In-Memory SQL databases. Data management and data retrieval is a key aspect of a successful business.

So today I am going to write about SQLFire ? It is the latest In-Memory/NewSQL Database server from VMware. I have given a table of commercial NewSQL Database servers and open source NewSQL  DB servers.

 Key Commercial Players   geniedb mysql cluster mysql nuodb rethinkdb scaledb sqlfire voltdb
Open Source Databases  mysql clustervoltdb

As a leader of Virtualization technology VMware is also thinking in these direction and developed some DB servers which can scale in a cloud platform and take the advantages of virtualized infrastructure. Thus giving the scalability and performance required for modern cloud based applications.

Database Type VMware Products
Relational Database vFabric Data Director + vFabric Postgres
NoSQL vFabric GemFire
NewSQL,In-Memory vFabric SQLFire
Unstructured/Big Data None today but Grenplum from EMC can be considered

But What is In-Memory /NewSQL Database server ?

To answer this ,let’s consider the speed of memory vs speed of the physical disks. Consider these two facts.

1.The seek time for memory is approximately 200 ns.

2.The seek time of a physical disk is approximately 20,000,000 ns.

So it obvious that if data is present in the memory it can be retrieved 100000 times faster than the data present in the disk. As memory is not a constraint in modern severs  now a days and a server’s can have 1 TB of RAM or even more, so the complete Database can be present in the Memory thus increasing the latency and performance of data retrieval. And we can have a backup of the whole database in a disk for the data to be consistent and present after server restart or server failure.

Another advantage of these In-Memory database is  the distributed nature. The database can be distributed across different servers present across geographies so the data does not travel in the network  as it stays near to to the Application..

So SQLFire is such a Database Server where the whole database resides in the Memory of the Server and can be distributed across physical Servers and geographies ,and in the center of it actually there is a VMware GemFire Server, GemFire is a In-memory ,Highly scalable ,Object based, High Performance data platform.

GemFire  is a NoSQL  DB Server.It does not understand SQL Queries, so to query, insert, modify or retrieve any data you have to do it Programmatically.

Gemfire Plus Apache’s Derby combined to make  a DB Server which understand SQL and one can Query, insert, modify the data using simple SQL language.

In-Memory databases are getting popular day by day because of it low latency performance, fast data access is getting more critical in business.Also when it comes to Horizontal scaling in-memory databases perform and scale linearly.

Today I am going to have a setup for Distributed SQLFire on My Desktop. I am going to create 4 Virtual machines with 3 SQLFire Servers and one SQLFire Locator.

This is going to be a perfect demo of how SQLFire works in a distributed environment and how Database Servers can be dynamically added and removed from a cluster of In-Memory Database Servers.

I am going to use CentOS this time for the Distributed SQLFire Setup.

1.Lets create a Virtual Machine template with CentOS, Java and SQLFire installed on it.

First I ll install CentOS linux on the VM created using VMware Workstation.

Steps To Create a CentOS VM Template :

1. Download CentOS from centos.org or from one of the mirror site.

2.Create a VM with 1 vCPU ,2GB RAM and 1 Network interface.

3.Install CentOS on it.

4.Set the network adapter to Host only. (Network address 192.168.0.0)

5.Create a Sharable Directory on your Base Windows OS,I call it “G:\VMshare” ,so that you can copy your files from the base Windows OS and Linux VMs.

CreateVM

5.Download Java for Windows and Linux from Oracle. Download SQLFire from VMware,download SQuirreL SQL Client to execute DB Queries.

6.Copy Java installer for Linux OS and the SQLFire installer to the sharable Directory.

7.Copy the Linux Java Installer and SQLFire Installer jar file to your Home directory on CentOS.

Shared-folder

8. Install Java on CentOS

9.Set the PATH and CLASSPATH environment variables on your .bashrc file for java installation.

10.Set the Network for the VM. i.e 192.168.0.10

SQLFire1-IP

Set the hostname for the VM by editing the /etc/sysconfig/network file.

#vi  /etc/sysconfig/network

HOSTNAME=sqlfireserver1

Edit the /etc/hosts file and enter the hostname and IP address of each VMs to be created.

#vi /etc/hosts

192.168.0.10          sqlfireserver1

192.168.0.11          sqlfireserver2

192.168.0.12          sqlfireserver3

192.168.0.13          sqlfirelocator

11.Install SQLFire on the VM.

$java -jar SQLFire-jar-installer -file.jar

12.Setup the PATH variable for SQLFire.

13.Now shutdown the VM and make 3 Clones of the VM as shown in the screenshot.

clone

14.After you finished the cloning, power on the Cloned VMs and modify the hostname and the network address (IP address) of each VM.

HOSTNAME=sqlfireserver1

HOSTNAME=sqlfireserver2

HOSTNAME=sqlfireserver3

HOSTNAME=sqlfirelocator

#vi /etc/hosts

192.168.0.10          sqlfireserver1

192.168.0.11          sqlfireserver2

192.168.0.12          sqlfireserver3

192.168.0.13          sqlfirelocator

15.Ping test of each VM communication ,make sure all the VM communicates with each other.

16.At this point I ll recommend to disable the firewall on each VM ,as this is a Host only network so you are safe inside your host machine.

Now you are ready with 4 VMs with SQLFire installed on each VM.

4 VMs

Tomorrow I am going to write how to use these VMs and configure and Start the SQLFire Database server.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s