150 words agree or disagree to each questions
Q1 Discuss the three important steps of text analysis
A: There are three main steps in the processing of text (data) analysis. Parsing, search and retrieval, and text mining. The first step, “parsing” is to develop a structure that the text will be converted to. This could be any structure that the analyst deems necessary for the totality of the project. Text will come in a raw data that is unstructured. “The unstructured text could be a plain text file, a weblog, an Extensible Markup Language (XML) file, a HyperText Markup Language (HTML) file, or a Word document.” (EMC Education Services, 2015, p. 310) Text in an of its self is normally not structured in a way that can be used raw, so this phase is critical to the production of the future data.
The second step is the search and retrieval, which sounds like the physical search but this step is about developing the “specific words, phrases, topics, or entities” (EMC Education Services, 2015, p. 310). This builds a key term(s) list for the information that will be used in the library to find the information. This type of text indexing is the same thing that happens with webpages on the internet so that making web search possible. Webpages are indexed based on key words and topics so that when they are search the page is found. The list of indexing that this step creates needs to be double checked, and continually checked to make sure that it is pulling the right data for the project.
The third step in text analysis is Text mining. This is the physical process using what the last two steps created to gather the text in a controlled way. This step also takes that text and puts it through the analysis process to make sure that it is the correct information needed to answer the question/problem sets. Such analysis could mean the use of k-means, clustering, and/or classification, … (EMC Education Services, 2015, p. 310).
What is also noted in our reading this week that I found interesting is that in a problem set dealing with text data all three steps do not have to be used. It depends on the goal of the problem set and what the focus is.
References
EMC Education Services. (2015). Data Science & Big Data Analytics: Discovering, Analyzing, Visualizing and Presenting Data. Indianapolis, IN: John Wiley & Sons, Inc.
Q2. There are three critical steps in text analysis parsing, search and retrieval, and text mining (Dietrich et at., 2015). Let us examine each step separately. Parsing is the process that takes the unstructured text and imposes a structure for further analysis (Dietrich et at., 2015). For example, a company may collect customers’ information thru various sources such as websites, social media, and physical stores. Each source may save the data in a different format; the company may store the information from the website in Markup Language (HTML) files, the social media in text files, and the sore information in Extensible Markup Language (XML) files. In the parsing process, an Analyst would transform the data in different formats into a single structure. For example, an Analyst may decide to create a single table in XML foramt with the information from all three sources.
The next step, search and retrieval, is the identification of the documents in a corpus that contain search items such as specific words, phrases, topics, or entities like people or organizations (Dietrich et at., 2015). A corpus, in this context, is a collection of written texts used for statistical research. In other words, after an Analyst arranges all the data formats into a single structure, the next step is to look for those records containing keywords and separate them into an independent file for further study. For example, if Facebook wants to know what people are commenting about their service outage, it could collect comments on their website. Then, collect and separate those comments that contain the words outage, down, service, and interruption somewhere in the text, for further analysis.
The third step, text mining, uses the terms and indexes produced by the previous two actions to discover meaningful insights pertaining to domains or problems of interest. In other words, the third step is to analyze the data to uncover information about the issue. Let us continue with the Facebook example. The company could use clustering and classification techniques to determine what people are commenting about their service outage. For instance, it could use a modified k-means method to cluster text documents into groups, where each group represents a collection of records with a similar topic. This method may reveal if most comments are negative, indifferent, or comical. With the insight collected from the text data, the company could prepare a PR campaign to respond to customer concerns.
These are the three critical steps in text analysis. I hope that it helps to understand this week’s material better.
Vr
Rommel P. Blanco
References:
Dietrich, D. (Computer scientist), Heller, B., & Yang, B. (2015). Data science and big data analytics: discovering, analyzing, visualizing and presenting data. Wiley.
