ThoughtWorks
Compartilhada publicamente -Data munging is a challenging, high-level activity because data itself is not as simple as many believe. http://thght.works/1SQTIAx
There is a story going around about data science that you’ve surely heard. It's the statement that 80% of the work a data scientist does is collecting, cleaning and organizing data and that only 20% is their real specialty: building models and making discoveries.
Those who make this statement, often those selling something, usually claim what data science needs most of all is better tools for doing these so called data munging tasks to free up da...
7
3
Data munger or Data wrangler (see WikiPedia).
To convert data content from one form to another requires a specification for the source and the target as well as a conversion algorithm. Thus, the job of "Data munger" or "Data wrangler" is to first develop the source and target specifications, and then building the conversion algorithm based on them.
How one splits the job along the 80/20 rule is another matter. However, it is interesting to note that usage of data content from a source requires some form of specification denoting how each data element is structured, what it means, and so on. Thus, specifications are the first and foremost job of a data scientist--understanding not only how data can be organized, but how that organization impacts data access and bulk analysis.
Yet--everything begins with specifications. Not all specifications are created equally. Some specifications at the data element level are built on suppositions of high orders of organization, client needs, access technologies, and a host of other matters that weigh in on the needs of the data clients.
When one begins to see the breadth and depth of the matter, one stops asking about 80/20 splits at superficial levels and for superficial reasons.
To convert data content from one form to another requires a specification for the source and the target as well as a conversion algorithm. Thus, the job of "Data munger" or "Data wrangler" is to first develop the source and target specifications, and then building the conversion algorithm based on them.
How one splits the job along the 80/20 rule is another matter. However, it is interesting to note that usage of data content from a source requires some form of specification denoting how each data element is structured, what it means, and so on. Thus, specifications are the first and foremost job of a data scientist--understanding not only how data can be organized, but how that organization impacts data access and bulk analysis.
Yet--everything begins with specifications. Not all specifications are created equally. Some specifications at the data element level are built on suppositions of high orders of organization, client needs, access technologies, and a host of other matters that weigh in on the needs of the data clients.
When one begins to see the breadth and depth of the matter, one stops asking about 80/20 splits at superficial levels and for superficial reasons.
· Traduzir
Adicionar um comentário...
