1: Identify and describe the initial problem/s that the TWC Company faced for which they used the new big data technologies? Describe two other problems that could be solved using big data technologies.
With the benefits of big data such as high data velocity, data variety, data volume and data complexity; The weather company can improve their forecasting at a very fast rate. Below I will list and describe some initial problems which big data could solve:
Weather is changing - The weather is changing because the climate is changing. Unfortunately, TWC cannot control the weather; however, they can improve their ability to predict it. Improve weather forecasting - TWC needs to be able to provide accurate forecasting and other companies
…show more content…
Provide reasoning to support the use of the NoSQL database as the database of choice to solve the problem faced by TWC. Identify one strength and one weakness for each of the other three kinds of databases to solve the problem for TWC.
I used the book NoSQL for Mere Mortals® to summarized the below answers.
Column family stores: Strength: great way to distribute data globally with high availability. It performs great with very large amounts of data distributed over many machines. Weakness: column oriented databases will be significantly slower when handling weather transactions.
Document database: Strength: Flexibility with the schema and supports querying more efficiently. Weakness: there is no standard query syntax.
Key-values stores: Strength: Simplest and easiest to implement. However, one of the weaknesses is that it doesn't perform well when querying or updating a particular value.
Graph database: Strength: designed for data whose relations are well represented as a graph and has elements which are interconnected. Graph databases are well-suited to irregular and complex structures. Weakness: Relationships are stored at the individual record level and uses more
The tables in relational databases organize data in rows and columns, simplifying data access and manipulation. It is easier for manager to understand the relational model than put all data in one table. Besides, a relational database allows tables to be linked. And the linkage reduces data redundancy and allows data to be organized more logically. In a word, relational database is easier to control, more flexible, and more intuitive than approaches.
One advantage to using relational databases include ease of use due to query languages, and the ease of altering the structure due to data independence. In addition, there are no predefined set of relationships between data records in the relational
Technology companies are using big data to analyze millions of voice samples to deliver more reliable and accurate voice interfaces. Banks are using big data techniques to im- prove fraud detection. Health care providers are leveraging more detailed data to im- prove patient treatment. Big data is being used by manufacturers to improve warranty management and equipment monitoring, as well as to optimize the logistics of getting their products to market. Retailers are harnessing a wide range of customer interactions, both online and offline, in order to provide more tailored recommendations and optimal pricing.96
The author points out that although there are existing algorithms and tools available to handle Big Data, they are not sufficient as the volume of data is exponentially increasing every day. To show the usefulness of Big Data mining, the author highlighted the work done by United Nations. In order to further enhance the reader’s perspective, the author provided research work of various professionals to educate its readers about the most recent updates in Big Data mining field. The author further describes the controversies surrounding Big Data. The author has first provided the context and exigence by elaborating on why we need new algorithm and tools to explore the Big Data. The author used the strategy of highlighting the logos by mentioning the research work of different industry professionals, workshops conducted on Big Data and was able to appeal to connect to the reader’s ethos. The author also used pathos by urging the budding Big Data researchers to further dig deep into the topic and explore this area
Database system design can be extremely time-consuming as it takes sophisticated software to create and control it. The design process becomes less user-friendly as it takes a more extensive knowledge of how to use it. The standard of an excellent database is one, which is complete, integral, simple, understandable, flexible and
The invention of relational databases have brought a number of changes to the business world in which they operate specially for the businesses whose prime focus is on its customers, their likes and dislikes to win more market share. There is no such concept as “one size fits all” in using this technology, it varies from industry to industry. One thing may work for some businesses and may not work for others, therefore it is advisable that one should shop around before investing in any of the technologies because it is vital to find an industry-specific solution. One technique to narrow the search for industry-specific solutions is to find out what our competitors are using to gain more customer base.
This model is most common if it is compared with network and relational database because it can be manage by huge amounts of data for difficult projects.
1) As big data is emphasized by more and more people, and the techniques and analytical
Next, I will address the advantages of a relational model specific to ACME Global Consulting. Since Acme Global Consulting is a leader in providing software development solutions to other companies, they need to have an in-depth knowledge of their clients functional and data needs. Therefore, a relational model can provide great benefits such as: (1). Conceptual simplicity: the model is very simple and helps to simplify data needs of their clients, (2). Design implementation: the model will help the company to achieve structural and data independence i.e. any changes if made to structure of data will not affect accessibility of data, and (3) Ad hoc capability of query implementation: relational model provides a very easy and flexible way of data implementation and manipulation. In addition, a relational model helps to organize data in form of tables. A relational model is composed of three main components: (1), a set of domains and sets that represents data structure, (2). Integrity rules, which ensures data protection, and (3). Operations that will be carried on data i.e. process of data manipulation. The experts on relational models say that it is the most widely used and most evolved method of organizing data Today, all major database management systems assume that you will be working in some manner with the relational model (Date, 2012, p.
Challenges: As Marcos explained: “A relational database wasn’t satisfying our requirements about performance and simplicity, due the complexity of our queries.” To address this, Marcos’ team decided to use Neo4j, a graph database, for which category Neo4j is the market leader.
NoSQL DBMSs advantages and Comparison to Relational DBMSs The reason why NoSQL has been so popular the last few years is mainly because, when a relational database grows out of one server, it is no longer that easy to use. In other words, they don't scale out very well in a distributed system. All of the big sites that you mentioned Google, Yahoo, Facebook and Amazon (I don't know much about Digg) have lots of data and store the data in distributed systems for several reasons. It could be that the data doesn't fit on one server, or there are requirements for high availability. Here is a table showing comparison and advantages of NoSQL over relational
In this model, data is stored in sections of columns instead of rows. This is almost an inverse of a relational model. The names of the column need not be predefined, i.e. the structure isn 't fixed, which helps in great scalability and performance. Columns in a row are stored in order according to their keys. A super-column also might be used which is nothing but a column containing nested sub-columns.
The information in triplestores can be stored, accessed and manipulated much easier versus using a relational database exponentially expanding data management and storage
Designers require to compromise between optimizing query performance and maximizing query flexibility. Columnar databases use less disk space and are more efficient in their I/O demands than records-based data warehouses but force their own compromise between optimizing for new record insertion versus data selection and retrieval. The current paper gives an idea about advantages and disadvantages of Row and Column Oriented Database concepts. The row by row approach keeps information about an entity together. In the student example above, it will store all information about the first student and then all information about the second student and so on. The column by column approach keeps all attribute information together: all of the student name will be stored consecutively and then all of the student email, etc Both approaches are well designed and typically a choice is made based on performance expectation if the expected workload is based on entity (e.g., find a student, add a student etc) then row by row approach is preferable because all information regarding a particular student is stored together but if the expected workload tends to read per query only few attributed from many records (e.g., a query that finds most common email address domain), then column by column storage is preferable since irrelevant attributes for a particular query do not have to be access. The row by row approach is writing
Database research and associated standardization activities have successfully guided the development of database technology over the last four decades and SQL relational databases remain the dominant database technology today. This effort to innovate relational databases to address the needs of new applications is continuing today. Recent examples of database innovation include the development of streaming SQL technology that is 170 George Feuerlicht used to process rapidly flowing data (“data in flight”) minimizing latency in Web 2.0 applications, and database appliances that simplify DBMS deployment on cloud computing platforms. It is also evident from the above discussion that the relational