NOTE: This is the fourth post in our Windward learning series on the introduction to digital data. Missed the beginning of the series? Read it here.
The rise of the Internet for personal and business use has led to the development and adoption of new ways of sharing data. While this data may be accessed by a user or customer directly, it is more commonly used by other applications.
Many of these technologies are similar to file-based data but optimized for passing data efficiently over a network. To achieve that efficiency, the data format needs to be simple and easy to access programmatically.
Major Examples of Web-Based Data Sources
Amazon S3, Google Drive, Flickr and Salesforce.com
There are a large number of existing Web applications/platforms that have an API you can use to access your data. The companies that own the applications store and administer the data storage. When a user wants access to their data, they can do so through an Internet connection.
These applications range from personal data, like email and images, to corporate data like Salesforce customer data.
The RESTful interface is a standard web interface for working with data stored in a Web application or Web service. It is very easy to work with since access is standardized. All major programming languages can access a RESTful interface.
XML and JSON
XML and JSON are industry standard Web-based data formats. XML is also commonly used as a file-based data source with a very easy-to-read format.
JSON is similar to XML but even more optimized for efficiency. It has a very compact format to reduce bandwidth while still being very easy to work with in an application.
Pros of Web-Based Data Sources
Wed-based data source have many advantages:
- Lightweight to minimize response time and reduce bandwidth.
- The simple data structure makes it easy to work with programmatically.
- Easy to connect to with a simple and well documented API. This is especially important since a given applications will have widely different needs.
Cons of Web-Based Data Sources
There are several disadvantages as well:
- Generally requires a fixed schema for your data, meaning that you must know what you want in the API during development.
- Does not include security and other advanced features, similar to file-based data. The application developer must add them manually.
When to Use Web-Based Data Sources
- When you are passing pre-defined data over a network, whether it is a company network or the Internet.
- This is especially for data made available outside of a company. For example, a web application would have a Web-based API for data access.
When Not to Use Web-Based Data Sources
- Data that needs to be modified often, especially by multiple applications/users.
- Moderate to large amounts of data. Performance will be slow or even possibly fail altogether.
The Bottom Line
Web-based data sources are a very convenient way for an application developer to work with data stored over a network. The ability to incorporate data from Web services and applications greatly expands application functionality and can give you a competitive edge.
Get the Whole Digital Data Series
Stay tuned for our next post, a look at big data as a data source for specialized applications. And remember, you can receive the whole series in your email inbox by subscribing to our company blog:
Author: En-jay Hsu
En-jay is a software developer from Boulder, Colorado.
Other posts by En-jay Hsu