Storage articles

Looking for something else? Check the categories of Storage:

Apache Avro Apache Cassandra Apache Hudi Apache Iceberg Apache Parquet Apache ZooKeeper Delta Lake Elasticsearch Embedded databases HDFS MySQL PostgreSQL Time series

If not, below you can find all articles belonging to Storage.

zNode in Apache ZooKeeper

As already told, zNodes are a key part in Apache ZooKeeper. They store information shared among different servers directly (as binary data) or indirectly (as parent directories).

Continue Reading →

Introduction to Apache ZooKeeper

Usually Apache ZooKeeper works in the shadow of more exposed Big Data tools, as Apache Spark or Apache Kafka. However, its role is very important in system architecture.

Continue Reading →

Elasticsearch migration from 1.6 to 2.2

At the begin Elastcisearch 2.2.0 was realeased on February 2016. Because my POC project was frozen with 1.6, I decided to upgrade. But not without surprises and some code rework.

Continue Reading →

Serialization and deserialization with schemas in Apache Avro

After theoretical introduction to Apache Avro, we can see how it can be used.

Continue Reading →

Introduction to Apache Avro

Previously we learned why serialization frameworks can facilitate work in distributed systems, where data provide from several different sources. Now, it's a good time to discover some real tools used in serialization step. As told, the chosen tool is Apache Avro.

Continue Reading →

Introduction to serialization in Big Data

NoSQL solutions are very often related to the word schemaless. Sometimes the absence of schema can lead to maintenance or backward compatibility problems. One of solutions to these issues in Big Data systems are serialization frameworks.

Continue Reading →

Reverse nested aggregation in Elasticsearch

Aggregations are a really powerful Elasticsearch feature. Besides aggregations known from RDBMS, such as sum, min, max, count, they offer the possibility to apply aggregation on different levels. It's particularly useful with nested documents.

Continue Reading →

Parent-children relationship in Elasticsearch

Make links between entities is quite easy in relational databases. And it's not a trivial task in document databases, adapted to less normalized data storage. Elasticsearch is not the exception of this rule but it defines some mechanisms to support parent-children relationship between documents.

Continue Reading →

Locks in Elasticsearch

Concurrency issues in Elasticsearch are often provoked by the lack of ACID transactions support. However, the search engine provides some of locking mechanisms to deal with them.

Continue Reading →

Aggregations in Elasticsearch

Even if Elasticsearch is not relational system, it allows to aggregate results. This operation is very helpful if we want to group set of documents.

Continue Reading →

Routing in Elasticsearch

If you've been worked with PHP frameworks like Zend or Symfony, you are certainly familiar with the concept of routing which is based on redirection of HTTP request to appropriated controller. Elasticsearch has similar feature, by the way, also called routing.

Continue Reading →

Proximity matching in Elasticsearch

Elasticsearch and its idea of inverted index is a kind of magic infinitely deep hat in which we can hide millions of terms. However, sometimes these terms need to be analyzed with some logic, not just only as plain words. It's here where proximity matching comes with help.

Continue Reading →

Partial matching and ngrams in Elasticsearch

Elasticsearch search matches only terms defined in inverted index. So even if we are looking for only two first letters of given term, we won't be able to do it with standard match query. Instead of it we should use partial matching, provided by Elasticsearch in different forms.

Continue Reading →

Filtered queries in Elasticsearch

Queries in Elasticsearch can be executed not only against full-text searches. They can also be filtered. And in Elasticsearch world, filters mean another operation than queries.

Continue Reading →

Scoring and boosting in Elasticsearch

A subtle difference between filter and full-text search consists on scoring. It's score who distinguishes result corresponding to filter from how well result matches the query.

Continue Reading →

Aliases in Elasticsearch

Nobody is perfect and my name is not nobody. Elasticsearch mapping, as mappings of another storage engines, is rarely immutable. Because of that, index changes can provoke service downtime, according to size of reindexed data. But there are a trick to avoid this dead times.

Continue Reading →

Indexing documents in Elasticsearch

Retrieving documents in Elasticsearch shouldn't be possible without indexing. They are a intermediate layer between user and shards which store documents data.

Continue Reading →

Bulk queries in Elasticsearch

Elasticsearch is devoted to store big amount of data. Making some operations as indexing them can be costly. It's one of the reasons Elasticsearch adopted the same features as in the most of principal RDBMS, batch operations - in Elasticsearch known as bulk operations.

Continue Reading →

Queries in Elasticsearch

Using Elasticsearch without querying is a little bit strange activity. After all, the name of this document-oriented database is composed by "search" suffix.

Continue Reading →

Connection modes in Elasticsearch

Elasticsearch has a powerful RESTful web service. But it's only one from available methods to connect our application to server.

Continue Reading →