Harnessing Social Data - Connecting, Capturing, and Cleaning

The first step to realize the promise of social data, which we went through in Chapter 1, Introduction to the Latest Social Media Landscape and Importance, is by harnessing it. A proper harnessing strategy can help to remove obstacles to and expedite processing. As we saw in the last chapter, many sources of social data can be used through the Application Protocol Interfaces (APIs) of these platforms. However, the data coming from APIs is not readily usable for multiple cases, hence it requires several steps before the data is ready to be analyzed and then applied. Therefore, we have dedicated a chapter that explains in detail how to do this. We have briefly touched upon the technical notion of an API in the first chapter. Here we intend to go deeper into it and help you to understand its types and usage. We also want to delve into its advantages and disadvantages so that you know exactly what to expect.

In the first chapter, we have seen the impact of social media on society at large. One of the best events in social media has been the ability to grow through the use of platforms and APIs. The sharing of data on social media platforms to third-party developers has resulted in thousands of creative applications of this data. It not only is a source of great learning for people, but also innovative use cases that impact the lives of millions of people. APIs also allow the creators of social media platforms to focus and build their platforms to perfection and let the creative applications of their data be exploited by anyone interested in them.

In this chapter, we will cover the following topics:

• Introduction to APIs and web services
• Introduction to authentication techniques
• Parsing API outputs
• Basic cleaning techniques
• Basic MongoDB administration