Adoption of NoSQL Database
Market trends shaping the Database Technology space
Current Database Technology market trends include Big Data, Hadoop, NoSQL and many flavors of unstructured data storage.The common theme is the realization that exponential data growth rates in combination with traditional data storage techniques will become unmanageable.We are always searching for solutions that balance performance, cost and dependability. Data growth impacts all three of these primary objectives. This is the trend behind the “trends” of today and into the future.
Solid state storage can both suppress performance problems for poorly written applications and accelerate performance of well written code
Newer technologies address some significant challenges when compared to relational databases. For example, if an organization or application leverages a combination of structured andunstructured data like audio files, photos, emails, videos, or social media then Hadoop may be a good solution. The combination of open source software and ability to leverage commodity hardware can drastically reduce both initial and operating costs when compared to relational databases.
NoSQL databases can take it a step further and are particularly beneficial when data requirements are unknown and applications must be rapidly developed. Dynamic schemas allow applications to add new fields instantly which can significantly reduce the length of the software development life cycle. Major NoSQL database typesinclude document dbs, graph stores, key-value stores and wide column stores.The adoption of these NoSQL database types should not come as a surprise as relational databases struggle withthese exact functions.
The reality is all databasetechnologies have limitations whether old or new. Selecting the right tool for the job should always be at the forefront when incorporating technologies into your solutions or organization.
The Database Exigencies
Providing availability, capacity and performance while balancing the costs associated with them always has been and remainsthe challenge for Database services organizations. These challenges should be combated in multiple ways including hardware, software, configuration, resources, planning and last but never least, managing expectations.
Investing in the appropriate hardware whether you leverage a private or public cloud is paramount. Application and database performance can only be as good as the available I/O levels of the hardware on which it resides.So if you are looking at a Hadoop solution across dozens of commodity servers or a large relational database residing on a single server, the total I/O available to the application always matters.
Solid state storage in particular has changed the game in the performance arena. Total IOPs on solid state storage are many orders of magnitudes higher than spinning disk. The sticker shock can be offsettinginitially but the benefits are extremely cost effective when compared to performance improvement alternatives like rewriting, re-architecting, or pausing new development to fix poorly performing applications. Solid state storage can both suppress performance problems for poorly written applications and accelerate performance of well written code.
Leveraging appropriate software is also a key weapon in the reliability and performance arenas. For example, proper monitoring software is vital to maintaining SLAs and proactively identifying inconsistencies within the data architecture and infrastructure. Compression and deduplication software orappliances can provide monumental cost savings in storage for both live data and backups.
From the configuration and resources perspective, maximizing memory usage significantly improves application performance in addition to reducing stress on other components of the infrastructure. Having the right engineers with experience in developing and configuring enterprise software, infrastructure and databases cannot be overlooked. This is often discounted and ultimately supersedes the much heavier investments made which ultimately affect the bottom line.
The Hype around NoSQL Technology
NoSQL databases can be cost effective and perform well. If you list all the functions that a relational database struggles with, you will see a list of the most popular NoSQL database types. These include document dbs, graph stores, key-value stores and wide column stores. The flexibility of NoSQL is particularly compelling since the data is not highly structured unlike relational databases. This aspect aligns well with an ever increasing ADHD world where we need it now and faster.
Enterprise thinking in mind, NoSQL is not a replacement for relational databases. It is a supplement. There are very important functions that the structure and design of a relational database provide that cannot be disregarded. Transactional consistency would be the first and only factor if deemed necessary for a solution.Ultimately, the ideal enterprise will have the best in class tools and solutions in place for the functions it is required to perform.
Ensuring Data Availability
Data availability is fundamental. That said, prioritizing required availability levels across the enterprise is a critical to success and managing the bottom line. High availability is expensive, resource intensive and adds complexity to almost every activity. The ancillary impact of high availability solutions is often overlooked. An organizations overall agility, flexibility and velocity will be affected by the level of data availability implemented.Establishing a tolerance for unavailability is a good step towards building the priority inventory. Another approach is to leverage components of a high availability solution for other purposes for example read only copies of data that can be used for basic reporting.
There is no replacement for experience, research and relationships. Field services will evolve just like almost everything else.