It takes time, money, and patience to collect research data. One of the ways to avoid collecting data yourself, is to use data which has already been gathered by other researchers. The number of data collections available is growing steadily, as is the diversity of data.
- The most comprehensive index of Australian research data is available via Research Data Australia (RDA). You can search RDA by subject, related organisation, or by the names of individual researchers.
- Government data from the Commonwealth, State and local governments can be searched at data.gov.au .
- Another source of Australian data is the Australian Bureau of Statistics (ABS). You can access ABS statistics through the USQ Library
- Trove brings together content from libraries, museums, archives and other research organisations and is maintained by the National Library of Australia.
- Australian ASX Financial data can be found using the DatAnalysis database, also available via USQ Library
There are a growing number of data indexes worldwide. Try some of the indexes listed below. [Please remove Science.gov from this list]
- DataCite a registry of datasets that searches across many disciplinary repositories
- FigShare a free cloud-based service for researchers to upload their data for discovery and access
- CODATA: International Council for Science: Committee on Data for Science and Technology
- OAIster: Harvest metadata from world-wide library repositories, including downloadable archival material
- World Bank Data contains free, open access data about development in countries around the world
- The United Nations Environment Programme Environmental data explorer is a directory of environmental data, including geospatial data sets
- Amazon Web Services (AWS) provides a centralized repository of public data sets, including the Human Microbiome Project Data Set which contains 9,811 sequence datasets for over 1,128 microbial genomes.
- European Union Open Data Portal allows access to census data from European Union institutions
- Data.gov is a portal to all the US government data available online
- The CIA’s World Factbook contains information on the history, people, government, economy, energy, geography, communications, transportation, military, and transnational issues for 267 world entities.
- Google Public data explorer houses a lot of data from world development indicators, OECD and human development indicators, mostly related to economics data and the world.
- Yahoo finance is a popular source of financial and market data
Much of the Social data available has been scraped for corporate and commercial interests, so many of the information available requires a subscription or payments of some kind. If you need social data for your research, you may need to factor this into your grant proposals.
If you are a capable programmer, the best place to get social data for an application programming interface (API) is the site itself: Instagram, GetGlue, Foursquare, pretty much all social media sites have their own API’s. Here are more details on the most popular ones.
- Twitter: Access to the Twitter API or check out out PeopleBrowsr, Gnip, DataSift, Infochimps.
- Foursquare: They have their own API and you can get it through Infochimps, as well.
- Facebook: The Facebook graph API is the best resource for Facebook.
- Face.com: A great tool for facial recognition data.
You may also want to try
- Google Trends: A free tool which scrapes the web for mega-trends
- Programmable Web: A really useful resource to explore API’s and also mashups of different API’s.
- Infochimps have a data marketplace that offers thousands of public and proprietary data sets for download and API access, in a wide range of categories, from historical Twitter and OK Cupid data, to geo locations data, in different formats. You can even upload you own data if you like.
- Qlik DataMarket is a commercial site where you can subscribe to explore data related to economics, healthcare, food and agriculture, and the automotive industry.
- Junar is a data scraping service that also houses data feeds.