Managing And Securing Unstructured Data

Chapter 7 discusses compression algorithms. Compressions are used often and sometimes we may not even be aware of it. The items we download or upload may be compressed in order to save bandwidth. Chapter 8 discusses the fundamental algorithms underlying databases (MacCormick, 7). This chapter emphasizes the techniques used to achieve consistency and to ensure that databases never contradict each other. Chapter 9 discusses the ability to ‘sign’ an electronic document digitally (MacCormick, 7). Chapter 10 discusses algorithms that would be considered great if it existed.

Computer databases are electronic filing system and are usually accessible on a computer network. The benefit of this method is that file access is so easy and quick by using a search process. Faster data access time can increase the productivity of managers and other employees who use data on a regular basis. Another benefit of such a system is that since electronic data is easy to backup in multiple locations, reducing the potential of a permanent data loss. The disadvantages of the electronic file system is that it prone to security since the computers are linked together on a network and hackers can get unauthorised access to your data. Another disadvantage is that it is costly to set up.

The relational model, which uses predefined tabular relations to store data, has remained the preeminent model for data storage since it was first implemented in the early 1980s. However, due to the proliferation of the Internet, today data flows in and out of organizations quickly, and most of this data is in a semi-structured state that is designed for communication over http. It is difficult to fit this complex data into a flat two dimensional array. For that reason, it is imperative that companies have the ability to store data in a semi-structured format compatible with modern network communications as well as various platforms and devices. The market has realized this and responded with document stores that support formats,

Data is either structured or unstructured. Where structured data is simply the data that can be stored in tables and these tables can be stored in conventional databases, while unstructured is a new type of data coming from tweets, likes, comments, text messages, google searching, photos, videos, Global

We will discuss in this paper the data warehousing and the online processing of data. We will describe the best ways to manage the data and the difficulties that you could face. Also we will talk about how can we solve or reduce these difficulties.

Additionally, social networking website Facebook, stores approximately 40 billion photos in total. (“Data, data everywhere”, 2010) Besides enormous data that generated from daily operational company transactions and social networks, the price drop of the data storage is also a strong factor triggering the fever of “Big Data”. For example, Google Drive - a cloud based data storage service – had a price drop of approximately 80% from March 2014. This price drop is considered a marketing approach to attract more computer users to adopt Google’s cloud service, which provides a more convenient and efficient way to access and store daily-used files. Although emerge of enormous data provides us opportunities to conduct further investigation and benchmarking, valuable information are not fully extracted and the potential power of using “Big Data” is undermined. In order to achieve thoroughly extraction of useful information from databases, many professionals in the academic field devoted into the study of data analysis and identified two of the most important drawbacks of traditional data analysis, which lacks of predictability and is less flexible in scalability.

Document Store or “Document-oriented database” is a data model within the NoSQL family, made for storing, retrieving, and managing document based information. The concept revolves around documents containing large amounts of data. A variety of documents are accepted, from there they are encapsulated into an internal format, and

In every moment of our lives, we handle a tremendous amount of large data, this large data will become a store of values if we could be turned it into searchable information involving analysis steps. The big challenge is that 90% of the large data are unstructured data which is growing faster than structured data, unstructured data as a data warehouse come without any predefined data structure and not appropriate with any relational database schema.

A non-profit might want to periodically store it's financial information this way for legal reasons. A university might want to store graduation records this way in case an accident destroyed the physical records. Businesses might want unalterable records to prevent tampering that could cover up a fraud.

Do you have an environment like Hadoop to handle unstructured data from social or IoT devices?

The modern RDBMS advancements are not capable of supporting unstructured information with ideal space necessity. The plan winds up plainly mind-boggling and is henceforth troublesome for designers. The requirement for unstructured information administration is so annoying with conventional RDBMS arrangements (Big data in financial services industry: Market trends, challenges, and prospects 2013 - 2018). Moreover, RDBMS turns out to be an exorbitant answer for creating light-footed web applications with direct information investigation necessities. NoSQL is developing as a proficient possibility in this situation, which connects the issues related with RDBMS innovation. The market development can credit to creative dispatches of NoSQL arrangements, and collective endeavors by NoSQL sellers and clients. The endeavors of organizations, to enhance their market offerings, are creating the request of NoSQL, as a back-end bolster (Big data in financial services industry: Market trends, challenges, and prospects 2013 - 2018). The emergence of agile software development is creating the demand for NoSQL (Big data in financial services industry: Market trends, challenges, and prospects 2013 - 2018). They offer users much more avenues to accept data in many different forms. NoSQL is adaptable as SQL but offers many more uses that can apply to many organizations.

Unstructured data are the data that do not have a recognized structure, they usually contain large amount of Texts. These types of data increase the level of difficulty to

As there is a rise in data volumes, the manageability of data and storing these huge volumes of data became a cause of concern to most of the organizations. It was during this period when Number of SQL or more popularly NoSQL was introduced, to process these large amounts of data efficiently and effectively. For this purpose, various Data Store categories were developed, based on the different data models. Some of the categories are:

Type of Data: A data could be broadly classified in to three types viz. structured data, unstructured data and semi-structured data. Structured data are those which reside in a fixed field within a record or a file such as customer’s name, address, phone number etc. Unstructured data refers to information that is not organized in a pre-defined manner. Examples of Unstructured model include images, videos, word presentations etc. It is estimated that unstructured data occupies almost 80% of all available data in organizations. Semi-structured data are those that do not have a strict data model. For example, an email contains both structured data such as sender’s name, recipient’s name and subject and also unstructured portion such as attachments and body of the letter. (http://www.webopedia.com/TERM/S/structured_data.html).In an integrated, 3-tier architecture such data are handled by XML as data traverses through various layers via an Enterprise Service Bus (ESB).Thus, choosing a database requires the type of data to be taken in to consideration .

Heterogeneity, scale, timeliness, complexity, and privacy problems with Big Data impede progress at all phases of the pipeline that can create value from data. The problems start right away during data acquisition, when the data tsunami requires us to make decisions, currently in an ad hoc manner, about what data to keep and what to discard, and how to store what we keep reliably with the right metadata. Much data today is not natively in structured format; for example, tweets and blogs are weakly structured pieces of text, while images and video are structured for storage and display, but not for semantic content and search: transforming such content into a structured format for later analysis is a major challenge. We will find out a way how structured big data can be transformed into unstructured data to increase the performance. Storage price trends have shown that now a days it’s not a big deal to afford storage for big un structured data. As far as performance is concerned, big data