Storage articles

Looking for something else? Check the categories of Storage:

Apache Avro Apache Cassandra Apache Parquet Apache ZooKeeper Elasticsearch HDFS MySQL PostgreSQL Time series

If not, below you can find all articles belonging to Storage.

Deletes in Apache Cassandra

Keeping old data eternally takes place and makes reads longer. Apache Cassandra is not an exception and has a mechanism to remove data.

Continue Reading →

Example of data consistency in Apache Cassandra

Previously we've presented theory of data consistency in Cassandra. Now it's a good moment to show some examples of consistency levels.

Continue Reading →

Data consistency in Cassandra

Distributed data brings a new problem to historical standalone relational databases - data consistency. Cassandra deals with this problem pretty nice with its different consistency levels.

Continue Reading →

Data organization on disk in Apache Cassandra

Until now we're working with Cassandra without looking on what happens. It's a time to be a little bit more curious.

Continue Reading →

Data part in Apache Cassandra

The previous article introduced us to Apache Cassandra by presenting vaguely its main concepts. This article focuses more in details on data topics.

Continue Reading →

Introduction to Apache Cassandra

After some articles about data ingestion and serialization in Big Data applications, it's time to start to learn about storage. This part begins with Apache Cassandra.

This article presents basic concepts of Apache Cassandra. In the first part it tries to explain architecture and general concepts of this solution. The second part is focused more on developer topics and it describes some main points about data organization.

Continue Reading →

Watches in Apache ZooKeeper

A lot of programming tools implement event-driven approach. Apache ZooKeeper isn't an exception for this rule with its system of watchers.

Continue Reading →

ACL in Apache ZooKeeper

Apache ZooKeeper is very often compared to distributed file system. Because each file system has a feature to deal with file permissions, ZooKeeper, as a kind of file system, can't be different.

Continue Reading →

Asynchronous operations in Apache ZooKeeper

Sometimes network latencies can slow down the communication between Apache ZooKeeper and its client. It's one of the reasons of possible use of asynchronous operations for zNodes manipulations.

Continue Reading →

Manipulate zNodes in Apache ZooKeeper

Until now we've seen how to create zNodes. But creation is not the single thing that Apache ZooKeeper does.

Continue Reading →

Session in Apache ZooKeeper

Client connects to ZooKeeper server and maintains a session. There are several things to know about ZooKeeper sessions and we'll explore them in this article.

Continue Reading →

zNode in Apache ZooKeeper

As already told, zNodes are a key part in Apache ZooKeeper. They store information shared among different servers directly (as binary data) or indirectly (as parent directories).

Continue Reading →

Introduction to Apache ZooKeeper

Usually Apache ZooKeeper works in the shadow of more exposed Big Data tools, as Apache Spark or Apache Kafka. However, its role is very important in system architecture.

Continue Reading →

Elasticsearch migration from 1.6 to 2.2

At the begin Elastcisearch 2.2.0 was realeased on February 2016. Because my POC project was frozen with 1.6, I decided to upgrade. But not without surprises and some code rework.

Continue Reading →

Serialization and deserialization with schemas in Apache Avro

After theoretical introduction to Apache Avro, we can see how it can be used.

Continue Reading →

Introduction to Apache Avro

Previously we learned why serialization frameworks can facilitate work in distributed systems, where data provide from several different sources. Now, it's a good time to discover some real tools used in serialization step. As told, the chosen tool is Apache Avro.

Continue Reading →

Introduction to serialization in Big Data

NoSQL solutions are very often related to the word schemaless. Sometimes the absence of schema can lead to maintenance or backward compatibility problems. One of solutions to these issues in Big Data systems are serialization frameworks.

Continue Reading →

Reverse nested aggregation in Elasticsearch

Aggregations are a really powerful Elasticsearch feature. Besides aggregations known from RDBMS, such as sum, min, max, count, they offer the possibility to apply aggregation on different levels. It's particularly useful with nested documents.

Continue Reading →

Parent-children relationship in Elasticsearch

Make links between entities is quite easy in relational databases. And it's not a trivial task in document databases, adapted to less normalized data storage. Elasticsearch is not the exception of this rule but it defines some mechanisms to support parent-children relationship between documents.

Continue Reading →

Locks in Elasticsearch

Concurrency issues in Elasticsearch are often provoked by the lack of ACID transactions support. However, the search engine provides some of locking mechanisms to deal with them.

Continue Reading →