Looking for something else? Check the categories of Storage:
Apache Avro Apache Cassandra Apache Hudi Apache Iceberg Apache Parquet Apache ZooKeeper Delta Lake Elasticsearch Embedded databases HDFS MySQL PostgreSQL Time series
If not, below you can find all articles belonging to Storage.
Client connects to ZooKeeper server and maintains a session. There are several things to know about ZooKeeper sessions and we'll explore them in this article.
As already told, zNodes are a key part in Apache ZooKeeper. They store information shared among different servers directly (as binary data) or indirectly (as parent directories).
Usually Apache ZooKeeper works in the shadow of more exposed Big Data tools, as Apache Spark or Apache Kafka. However, its role is very important in system architecture.
At the begin Elastcisearch 2.2.0 was realeased on February 2016. Because my POC project was frozen with 1.6, I decided to upgrade. But not without surprises and some code rework.
After theoretical introduction to Apache Avro, we can see how it can be used.
Previously we learned why serialization frameworks can facilitate work in distributed systems, where data provide from several different sources. Now, it's a good time to discover some real tools used in serialization step. As told, the chosen tool is Apache Avro.
NoSQL solutions are very often related to the word schemaless. Sometimes the absence of schema can lead to maintenance or backward compatibility problems. One of solutions to these issues in Big Data systems are serialization frameworks.
Aggregations are a really powerful Elasticsearch feature. Besides aggregations known from RDBMS, such as sum, min, max, count, they offer the possibility to apply aggregation on different levels. It's particularly useful with nested documents.
Make links between entities is quite easy in relational databases. And it's not a trivial task in document databases, adapted to less normalized data storage. Elasticsearch is not the exception of this rule but it defines some mechanisms to support parent-children relationship between documents.
Concurrency issues in Elasticsearch are often provoked by the lack of ACID transactions support. However, the search engine provides some of locking mechanisms to deal with them.
Even if Elasticsearch is not relational system, it allows to aggregate results. This operation is very helpful if we want to group set of documents.
If you've been worked with PHP frameworks like Zend or Symfony, you are certainly familiar with the concept of routing which is based on redirection of HTTP request to appropriated controller. Elasticsearch has similar feature, by the way, also called routing.
Elasticsearch and its idea of inverted index is a kind of magic infinitely deep hat in which we can hide millions of terms. However, sometimes these terms need to be analyzed with some logic, not just only as plain words. It's here where proximity matching comes with help.
Elasticsearch search matches only terms defined in inverted index. So even if we are looking for only two first letters of given term, we won't be able to do it with standard match query. Instead of it we should use partial matching, provided by Elasticsearch in different forms.
Queries in Elasticsearch can be executed not only against full-text searches. They can also be filtered. And in Elasticsearch world, filters mean another operation than queries.
A subtle difference between filter and full-text search consists on scoring. It's score who distinguishes result corresponding to filter from how well result matches the query.
Nobody is perfect and my name is not nobody. Elasticsearch mapping, as mappings of another storage engines, is rarely immutable. Because of that, index changes can provoke service downtime, according to size of reindexed data. But there are a trick to avoid this dead times.
Retrieving documents in Elasticsearch shouldn't be possible without indexing. They are a intermediate layer between user and shards which store documents data.
Elasticsearch is devoted to store big amount of data. Making some operations as indexing them can be costly. It's one of the reasons Elasticsearch adopted the same features as in the most of principal RDBMS, batch operations - in Elasticsearch known as bulk operations.
Using Elasticsearch without querying is a little bit strange activity. After all, the name of this document-oriented database is composed by "search" suffix.
