Big Data

The term big data is used for large volume of the data which includes both the structured and the unstructured data. Big data inundates the business on daily basis. But here the amount of data is not important but the thing that matters is what organisation does with the big data.

One potential application of big data is SEC (Security Exchange Commission). SEC monitors the activity of financial market by using big data. In financial markets illegal activities are catch by SEC using the natural language processors and network analytics

For the trade analytics which is used in a high frequency trading, sentiment measurement, analytics of pre-trade decision support, predictive analytics and many others, the big data is used by retail traders, hedge funds, and big banks in financial market.

With the real-time big data a business can detect frauds and errors quickly. Also businesses develop effective strategies using big data in a very less time. But the disadvantage is that the current tools which are used are not capable of handling the real-time analysis.

The Age of Big Data

In the 21th century, among all the greatest power sources, Big Data is one of them. ‘The Age of Big Data’ is a 58 minutes BBC documentary which follows the people who mining Big Data. Varying uses which can be made with the help of the enormous amount of data available in the databases are examined in the video. This video shows the police officers of LADP, who can forecast the crime to happen most likely by using Big Data. In this video a scientist from London is shown who use maths to make millions and also an astronomer from South Africa who is preparing to catalogue the whole cosmos.

Data mining: Big Data’s Increasing Challenge and Payoff

On the daily basis, the data generated is incredibly of large volume. The information is hide in this which can be valuable on economic, environmental, political and social level. But the need is to mine the data to extract the information. The information for analysis is available in huge amount and grows with the every Facebook post and bar-code scan.

Super-computing: Power of visualization

In this digital age, astonishing ways are emerging to share the scientific data. In this 27 minutes film ‘super-computing: Power of visualization’, remarkable growth of supercomputing are explored and its impact on oceanographic, medicine, law enforcement, air traffic control and other fields are examined. It showcase the real-world applications of dynamic visualization technology, capabilities of cellular level medical imaging, three dimensional interpretation of the cities accurate to square foot and prompt collaboration between the hemisphere apart laboratories. Also the Jaron Lanier, a pioneer of virtual reality and other innovators commented and highlighted the human factor need in the supercomputers’ age.

OLTP and OLAP

IT system can be divided into OLTP (transactional) which provides the source data to the data warehouses and OLAP (analytical) which helps to analyse this data. OLTP stands for On-line Transaction Processing in which short on-line transactions of very large amount are characterized. OLAP stands for On-line Analytical process in which comparatively less volume of transactions is characterized. In OLAP aggregations are involved and mostly the queries are complex and response time is considered an effective measure. In OLTP, processing the fast queries, maintaining the data integrity and effectiveness is measured by the number of per second transactions. Detailed and the current data is available in the OLTP database.

Star and Snowflakes Schemas

To organize entire data warehouses or data marts, relational databases by the way of Star and Snowflakes Schemas are used. Dimension tables are used in the fact table to describe the data aggregated in both of them. The difference in them is that dimension table is normalized in snowflakes schema and the query is complex more in it. Star schema is best to use in data marts whereas snowflake is used in data warehouses. The data hierarchy model in snowflake schema is shown in primary key while in star schema only foreign keys are present in the fact table. Snowflake model is useful for dimension analysis whereas star schema is best for metrics analysis.