Big data is a term for data sets that are so large or complex that traditional data processing applications are inadequate. The challenges associated with such huge amounts of data include analysis, capture, data curation, search, sharing, storage, transfer, visualization, querying, updating and information privacy. Big data often could be used for preparing the data set for predictive analytics, user behavior analytics, or certain other advanced data analytics methods that extract value from data which is useful for various purposes. Accuracy in big data may lead to more confident decision making, and better decisions can result in greater operational efficiency, cost reduction and reduced risk.
Use cases surrounding Big data
To quote a few, analysis of data sets can be used in finding correlations in :
- Business trends
- Health Sector
- Combatting Crime rates
- Weather patterns
- Meteorology
- Business Informatics
- Finance
- Simulations and research
A brieflook at the hardware setup
To handle the ginormous amount of data, traditional server infrastructure and relational databases are not enough. They require a special breed of data bases constructs such as (MongoDB etc) and several massive swarms of servers (many thousands of servers) running even more massive parallel softwares depending upon capabilities of the users and requirements for analytics.”For some organizations, facing hundreds of gigabytes of data for the first time may trigger a need to reconsider data management options. For others, it may take tens or hundreds of terabytes before data size becomes a significant consideration.”
