NameNode (NN) NN holds the metadata of the files in HDFS, maintains the entire metadata in RAM It is important to run NN from a machine that has lots of RAM at its disposal. The higher the number of files in HDFS, the higher the consumption of RAM In case the namenode daemon is restarted Read fsimage…
In this tutorial, I will write down step-by-step guide on how to install ElasticSearch on CentOS. Well, if you do not know what is ElasticSearch, then it can be defined as follows: Elasticsearch is a search server based on Lucene. It provides a distributed, multitenant-capable full-text search engine with a RESTful web interface and schema-free…
This will add a quick note on how to install necessary Hadoop ecosystem by Installing Apache BigTop for Hadoop Development. Remember that we need a 64-bit OS to be able to run Big Data project. First, add a new repo. The latest BigTop repo (0.8.0) is located at http://www.apache.org/dist/bigtop/bigtop-0.8.0/repos/ and we just need to add appropriate repo…
Recent time, I supporting a development team on customizing an intensive social network website with integrated IPB forum software. The main problems that the team faced are (1) very slow on accessing the website, and (2) the db is often crashed in the peak time (about 1000 online users). There are some checklists which might help…
The first part of this article is copied from http://www.if-not-true-then-false.com/2010/install-mongodb-on-fedora-centos-red-hat-rhel/ since we will not reinvent the wheels 🙂 What is MongoDB? MongoDB (from “humongous”) is a scalable, high-performance, open source, schema-free, document-oriented database. Written in C++. MongoDB bridges the gap between key-value stores (which are fast and highly scalable) and traditional RDBMS systems (which provide structured schemas…
NoSQL databases are often compared by various non-functional criteria, such as scalability, performance, and consistency. This aspect of NoSQL is well-studied both in practice and theory because specific non-functional properties are often the main justification for NoSQL usage and fundamental results on distributed systems like the CAP theorem apply well to NoSQL systems. At the same time, NoSQL…
In 2010, when the world became enchanted by the capabilities of cloud systems and new databases designed to serve them, a group of researchers from Yahoo decided to look intoNoSQL. They developed the YCSB framework to assess the performance of new tools and find the best cases for their use. The results were published in…
Whenever analysts or journalists assemble lists of the top trends for this year, “big data” is almost certain to be on the list. While the catchphrase is fairly new, in one sense, big data isn’t really a new concept. Computers have always worked with large and growing sets of data, and we’ve had databases and…
1. Quick Comparison While SQL databases are insanely useful tools, their monopoly in the last decades is coming to an end. And it’s just time: I can’t even count the things that were forced into relational databases, but never really fitted them. (That being said, relational databases will always be the best for the stuff…