Back to top

Open Data - the Why?

Why is Open Data important?

For government to make evidence-based decisions on policy, budgets and public services, they need data. For citizens to engage with government and hold them accountable, they need data. For civil society to target their programs and measure development goals, they need data. For companies to develop new innovations and make investment decisions, they need data. For Nepal’s youth to do their studies and be able to fully realize their potential in society, they need data.
 
Data on lots of topics already exists in Nepal – health, education, population, economy, environment, etc., however much of it is kept private or shared in formats that prevent the data being used freely. Sharing data in open format means that everyone can get equal access to important evidence that can be used in lots of different ways to improve people’s lives.
 
Nepal is facing many pressing development challenges related to poverty, accountability, transparency, economic development, and social inclusion. To address these challenges, the use of reliable, accessible evidence to inform decision-making, monitor progress, and evaluate development outcomes is essential. The term “open data” refers to the format in which this evidence should be produced, shared and used.
 
Progress in the sharing and use of data for development in open format in Nepal has been made over recent years – concepts of data transparency and rights to data have become increasingly known to champions both inside and outside government, and there is growing recognition of open data as an important tool for development. The government has taken important steps to open up its data - releasing open data on topics like foreign aid money, government purchasing, and earthquake related needs. Non-government groups are working to encourage and support the opening up of data, and coming up with innovative ways to increase awareness about data and the benefits of opening it up as well as building capacity to use the data. Open Nepal is a community of organizations and individuals who want to see better sharing and use of open data.

Helpful Resources

The Importance of Open Data – World Economic Forum: https://www.weforum.org/agenda/2016/02/the-importance-of-open-data

The Economic Benefits of Open Data – European Data Portal: https://www.europeandataportal.eu/en/highlights/economic-benefits-open-…

Benefits of Open Data – European Data Portal: https://www.europeandataportal.eu/en/using-data/benefits-of-open-data

Open Data in Developing Economics – The GovLab: http://odimpact.org/developingeconomies.html

Why should the government know about Open Data?

Open data can help governments to make informed decisions and policies. For example, open data on demographics can help governments to more effectively allocate resources based on the needs of different provinces and municipalities. Open data on health and education can help governments to identify key populations that require service delivery, and target the efforts of development partners. Open data on budgets and foreign aid can help governments to track financial flows and ensure commitments are met. With Nepal moving into the federal structure, data will play a critical role for governments at all spheres to understand one another and their needs, and to plan effective services and policy. Similarly, open data will allow governments as well as citizens to track progress of and evaluate the effectiveness of service delivery and policies in Nepal. Opening government data will lead to transparency that will also ensure public trust and faith in the government.

Over the past decade, open data as an idea has been gaining political relevance with its benefits potential for governance beginning to be seen around the world. Open government data is increasingly being recognized as the best practice around the world for transparency, accountability, efficiency and evidence-based decision-making. In Nepal too, the idea of open data and open government is beginning to gain in importance. The federal government is the most important supplier and user of data and can play an integral role in pioneering the sharing and use of open data.

Why should civil society know about Open Data?

Open Data can help civil society organisations to better understand ground realities and respond to them effectively. For example, open data from surveys of people’s needs, attitudes and practices can help civil society organisations set target for their activities. Open data on the work of civil society organizations can help others avoid duplication of efforts, and can help in monitoring and evaluating the effectiveness of development projects. In addition, open data from governments can help civil society to monitor government performance and hold them accountable.

There is huge potential for civil society to play a catalytic role in encouraging the supply and use of open data, and in demonstrating the value and potential of open data for Nepal’s development. Open Nepal is a group of non-government organisations which are trying to build Nepal’s open data ecosystem.

Why should the private sector know about Open Data?

Open government data can be used by businesses to improve business processes and enable their more efficient and profitable operations - businesses require easy access to routine information on key processes and official data to underpin their regular business procedures. For example, companies need to know about customs schedules, registration procedures, how to pay their taxes, labour markets, consumer markets, and about government procurement tenders, etc. Open government data can be also used by business to enable new data-driven products and services – businesses can use open data to create business opportunities through the analysis and integration of open data (e.g. mapping data, data on market prices, data on consumers) in products and services.

Open Data - the What?

What is Open Data?

The term open data simply refers to raw data that is made available to the public in a way that can be accessed and processed by computers. By disseminating data in this way, data can be freely used, analyzed and shared by anybody who might want to use it for any purpose. In essence, open data is a tool for delivering greater equality of access to information and greater opportunity for innovation. By enabling improved availability and accessibility of information, open data can empower government, citizens, business and civil society to engage with public issues and make better informed decisions to improve their lives.

Two elements commonly seen as essential for data to be considered “open data"1:

  • Technical: Data must be technically open by being published in electronic formats that are machine readable and non-proprietary. It should be made available on a public server without any restrictions for use.
  • Legal: Data must be legally open by being placed on a public domain or with liberal terms of use.

In Nepal’s context, making government, civil society, private sector, academic or scientific data freely available to the public online is an important step towards open data.

Helpful Resources

Open Definition: http://opendefinition.org/od/
Open Data Handbook: http://opendatahandbook.org
European Portal: https://www.europeandataportal.eu/elearning/en/module1/#/id/co-01
World Bank Open Data Toolkit: http://opendatatoolkit.worldbank.org/en/essentials.html

1. http://opendatatoolkit.worldbank.org/en/essentials.html

When is data “open data”?

For data to be considered open, it must be in a machine-readable format (e.g. JSON, XML, CSV), ideally free to download, and provided under terms that permit reuse, redistribution and intermixing with other datasets. A number of globally recognized principles have been developed that determine whether data is open. For example, the Sunlight Foundation states that government data shall be considered open if it that complies with the following principles2:

  1. Completeness: Datasets released are the entirety of what is recorded about a particular subject, including metadata, with the exception of personally identifiable information.
  2. Primacy: Datasets released are the primary data collected at the source, with the finest possible level of granularity, not in aggregate or modified forms.
  3. Timeliness: Datasets released are made available as quickly as necessary to preserve the value of the data.
  4. Ease of Physical and Electronic Access: Datasets released are as accessible as possible, though physical or electronic means.
  5. Machine Readability: Datasets are reasonably structured to allow automated processing, and shared in widely-used file formats that can be easily machine processed.
  6. Non-Discrimination: Datasets released are available to anyone, with no restrictions or requirement of registration.
  7. Use of Commonly Owned (Open) Standards: Datasets are available in a format over which no entity has exclusive control.
  8. Licensing: Datasets are not subject to any copyright, patent, trademark or trade secret regulation. Reasonable privacy, security and privilege restrictions may be allowed as governed by other statutes.
  9. Permanence: Datasets are available online in permanent archives, with appropriate version tracking over time.
  10. Usage Costs: Datasets are available at no or minimal charge to the user.

Helpful Resources

Sunlight Foundation: https://sunlightfoundation.com/policy/documents/ten-open-data-principle…
Open Data Charter: https://opendatacharter.net/principles/

2. https://sunlightfoundation.com/policy/documents/ten-open-data-principle…

What are some of the myths about Open Data?

There are a number of common misconceptions about open data.  

  • Open data does NOT mean that original data-points can be altered by the user – open data does not allow individual data points to be changed on the original source of publication, e.g. the host website or data portal.
  • Open data does NOT mean that personal data becomes public – the focus of government open data portals is non-personal data. Open data does not contain information about specific individuals, and identifying features can be removed before data is released publicly.
  • Open data is NOT a breach of national security – the data supplier, often a government, can choose which data they wish to make open. National security restrictions may apply to some types of data.
  • Open data is NOT necessarily expensive or difficult – Many off-the-shelf frameworks exist already to support data sharing, and modern cloud-based platforms can prevent the need for large-scale hard-ware investment.
  • Data presented in PDF reports or written as website text is NOT open – for data to be considered open it has to be shared in a machine-readable format.
What is the Data Revolution?

While efforts to improve the production, sharing and use of data have been taking place for many years, the need for improvements in data have recently received elevated attention on a global stage. In 2015, 193 member states of the United Nations agreed 17 global goals, known as the Sustainable Development Goals (SDGs), which commit every country to address the root causes of poverty and to ensure that development efforts leave no-one behind. At the heart of this new global development agenda is an acknowledgement that timely, usable data is critical to its implementation and monitoring. The term “data revolution for sustainable development” describes the transformational shift required in the way data is produced, shared and used for the planning, implementing and monitoring of development. This requires radical change in the way that data is harnessed to impact on development decision-making and accountability, with a particular emphasis on building a culture of usage. Open data is a critical part of the Data Revolution. The exponential increase in the volume and types of data will only be of value if it is available to the decision-makers that need it.

Helpful Resources

A World that Counts – United Nations: http://www.undatarevolution.org/data-revolution/, https://www.csis.org/analysis/harnessing-data-revolution-achieve-sustai….
Data Revolution and SDGs: https://www.innovations.harvard.edu/blog/data-revolution-and-sustainabl…
Nepal’s Emerging Data Revolution – Development Initiatives: http://devinit.org/post/insights-into-nepals-emerging-data-revolution/

What is the Open Government Partnership?

Around the world governments are taking important steps to promote more open, accountable and responsive governance. These aspirations are also reflected in Goal 16 of the 2030 Agenda. A global initiative that is supporting this trend is the 'Open Government Partnership' (OGP). The OGP has supported seventy countries, both in the developed and developing world, to implement plans, policies and practices that support government openness. Nepal is not currently a member of the OGP but is eligible to join.

The OGP is an international partnership of reformers in government and civil society, committed to making governments more transparent, responsive and accountable. To achieve these goals the OGP secures national commitments from governments to promote transparency, encourage civic participation, combat corruption and harness new technologies to strengthen governance. These national commitments aim to make government more open in order to improve public trust in government, promote investment climate and economic growth, and to mitigate the grievances that facilitate conflict and violence. An important feature of the OGP is that it brings governments and civil society organizations together as equal partners in developing national level reforms to promote open and transparent government.

Under the four principles of open government, OGP participating countries declare their commitment to increase the availability of information about governmental activities; support civic participation; implement the highest standards of professional integrity throughout their administrations; and increase access to new technologies for openness and accountability.

Helpful Resources

Open Government Partnership: https://www.opengovpartnership.org
Open Government Data (Book): https://opengovdata.io
Open Government Data: Right to Information 2.0 or its Rollback Version: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2152566
Open Nepal’s Briefing on Open Government Partnership – Case of Nepal: http://opennepal.net/resource/briefing-open-government-partnership-case…
Nepal: Open Government Partnership Readiness Assessment (Accountability Lab): http://www.accountabilitylab.org/wp-content/uploads/2015/12/nepalOGP-re…

What policy is there on Open Data in Nepal?

In Nepal, currently, there no specific open data policies or legal instruments have been adopted. However, in August 2017, the Government of Nepal’s National Information Commission submitted an Open Data Action Plan on Open Government Data to the Prime Minister’s Office. The acceptance of this by the Prime Minister’s Office highlights the intention of the Government of Nepal to support the open data agenda in Nepal.

Without specific legal instruments and policies on open data in Nepal, the open data agenda in Nepal is currently linked to the right to information of Nepali citizens. In Nepal, Article 4 of the Constitution recognizes right to information as a fundamental right of the citizens of Nepal. Every citizen has the right to demand and receive information on any matter of public interest, apart from sensitive information that have been classified as not disclosable. The Right to Information Act, enacted in 2007, legislated this right with the aim of making the functions of the state open, transparent, responsible and accountable to citizens in accordance with the democratic system. Nepal therefore already has a legal framework, and an independent oversight body (the National Information Commission) in place to ensure that information sharing by government exists in Nepal and that citizens can obtain information of public importance.

Open Government Data is aligned with Right to Information in many ways. They are both mechanisms that facilitate the availability of government information, however under the Right to Information mechanism the government responds to individual public requests for information, while under the Open Government Data mechanism the government proactively discloses raw data on a public platform. In Nepal, the Right to Information Act 2007 and the Right to Information Rules 2009 support proactive disclosure of information, and requires public agencies to proactively release information, maintain updated information, and appoint an information officer. While Right to Information will always be a critical facilitator of government transparency and accountability, the concept of Open Government Data can augment this process by enabling that transparency to be used for innovation and economic growth.

There are a number of other legal instruments that support the production, sharing and use of information, for example the Local Self Governance Act 1999, which mandates each of Nepal’s 75 district development committees to have an information and record center for collecting and maintaining local information.

Helpful Resources

Nepal’s Emerging Data Revolution – Development Initiatives: http://devinit.org/post/insights-into-nepals-emerging-data-revolution/
Open Government Data and the Right to Information -  Opportunities and Obstacles: http://ci-journal.net/index.php/ciej/article/view/952/954
Access to Government Information – Right to Information and Open Government Data Synergy: https://ac.els-cdn.com/S187704281404018X/1-s2.0-S187704281404018X-main…
Open Nepal’s Briefing on Open Government Partnership – Case of Nepal: http://opennepal.net/resource/briefing-open-government-partnership-case…
Open Nepal’s Briefing on Links Between Open Data and RTI: http://opennepal.net/resource/open-nepal-briefing-links-between-open-da…

What the (open) data ecosystem?

The term “data ecosystem” is used to describe the interactions and relationships between producers, users, owners and beneficiaries of data and information. The groups that make up the data ecosystem include government, civil society, private sector, academia etc, at all levels – local, national and international. Many of these groups act both as suppliers, users and beneficiaries of data. For a data ecosystem to be functional in meeting all stakeholder needs, there needs to be a reliable supply of good quality, accurate, timely, disaggregated and open data. The release of data needs to be driven by demand and there needs to be capacity and skills for people to use the data.

To communicate data demand to data owners, encourage data producers to share data, and turn the data released into information and knowledge that is useful and meaningful for society, there needs to be a community of “data infomediaries”. This includes data journalists, tech specialists, information freedom advocates, governance experts, social entrepreneurs, and accountability activists, etc. The work of these groups helps ensure data ecosystems are functional, inclusive, impactful and sustainable. Open Nepal is a community of such data infomediaries.

Helpful Resources

Open Data: Driving growth, ingenuity, and innovation – Deloitte: https://www2.deloitte.com/content/dam/Deloitte/uk/Documents/deloitte-an…
Nepal’s Emerging Data Revolution – Development Initiatives: http://devinit.org/post/insights-into-nepals-emerging-data-revolution/
Open Data Manual – Open Knowledge Nepal: http://odap.oknp.org/files/Open%20Data%20Book%20Manual.pdf

What are the different sources of Open Data in Nepal?

For information about the sources of open data in Nepal please visit the Open Nepal Data Hub

What is Open Licensing?

Licenses dictate what can and cannot be done with someone’s work by granting permissions for use or stating restrictions for use. For data to be considered legally open it must have an open license, which simply means a license that allows others to access, re-use and re-distribute a work with little or no restrictions.

Intellectual property rights often prevent third parties from using, reusing and redistributing data without explicit permission. Therefore, when planning to make data open it is important to add a license to make it clear that people are free to use your data legally.

Creative Commons Licensing means that anyone is able to use and redistribute the material in any medium or format for any purpose, even commercially. Sometimes these licenses require the need for attribution. This means giving appropriate credit by providing a link to the license and indication of any changes made.

Helpful Resources

Reusers Guide to Open Data Licensing: http://theodi.org/guides/reusers-guide-open-data-licensing
Open Definition Guide to Open Data Licensing: http://opendefinition.org/guide/data/
Conformant Licenses: http://opendefinition.org/licenses/
Making your Data Open, A Guide: http://opendatacommons.org/guide/

Open Data - the How?

How to open data that is in closed format?

Electronically available data is often not in an open or machine readable formats, but instead in closed formats such as pdf documents or HTML (web text). Also it can be poorly structured. This makes it difficult for people and programs to  work with the datasets, and therefore it becomes essential to extract the data – i.e. turn it from closed data into open data. Extracting data from websites and pdfs into open format, is also called data scraping or web scraping. There are a number of ways in which data can be extracted from a closed format to make it open, ranging from simple software to intensive coding.

Helpful Resources

Open Data Manual – Open Knowledge Nepal: http://odap.oknp.org/files/Open%20Data%20Book%20Manual.pdf
Data Extraction Tools: https://bbvaopen4u.com/en/actualidad/data-extraction-tools-beginners-an…
Web Scraping (Python): https://www.analyticsvidhya.com/blog/2015/10/beginner-guide-web-scrapin…

How to clean data, especially after scraping from closed format?

Data cleaning is the process of identifying and removing incomplete, incorrect, inaccurate and irrelevant data from a certain dataset. Cleaning data helps to tidy up the numbers, and also ensures that datasets are clear to understand and useful to work with. Data cleaning is also known as data scrubbing.

Helpful Resources

Data Cleaning Tools: https://www.datasciencecentral.com/profiles/blogs/5-data-cleansing-tools
Data Cleaning Tools: https://analyticsindiamag.com/10-best-data-cleaning-tools-get-data/
School of Data: Scraping: http://schoolofdata.org/handbook/courses/scraping/
Scraping Website using the Scraper extension for Chrome: http://schoolofdata.org/handbook/recipes/scraper-extension-for-chrome/
Data Journalism Handbook: http://datajournalismhandbook.org/1.0/en/getting_data_3.html
QuickCode – Python and R Data Analysis: https://quickcode.io

How to analyse and visualize Open Data?

After the data is scraped and cleaned, one needs to work with the dataset to find relationships and patterns, and add statistical or logical techniques to build, support or falsify arguments. This process is called data analysis, which is simply the process of evaluating data with statistical or logical reasoning. Once data is analysed, to communicate and present the data or findings in an easy to understand way through visuals, such as graphs or charts, is called data visualization.

Helpful resources

School of Data: Analysing data: http://schoolofdata.org/handbook/courses/analyzing-data/.
Data Visualization Tools: https://www.forbes.com/sites/bernardmarr/2017/07/20/the-7-best-data-vis…
Open Source Data Visualization Tools: https://blog.capterra.com/free-and-open-source-data-visualization-tools/
Data Analysis in Microsoft Excel: http://www.excel-easy.com/data-analysis.html
Data Analysis in SPSS:  https://students.shu.ac.uk/lits/it/documents/pdf/analysing_data_using_s…
Data Analysis in Google Spreadsheets: https://support.google.com/docs/table/25273?hl=en&page=table.cs&rd=1
Pitfalls while analyzing data: http://schoolofdata.org/handbook/courses/common-misconceptions/

How to publish data in an open format?

To publish data in an open format, first start with picking the dataset that you wish to publish and apply an open license to it. Choose the format for the data and make it available online for the public for use.

Helpful Resources

How to Open Data: https://okfn.org/opendata/how-to-open-data/
Choosing the right format for open data: https://www.europeandataportal.eu/elearning/en/module9/#/id/co-01
Open File Formats: http://opendatahandbook.org/guide/en/appendices/file-formats/
Open Data: What is it and why should you care: http://www.govtech.com/data/Got-Data-Make-it-Open-Data-with-These-Tips…

Open Data – Useful Resources From Nepal

Briefing: Open Data

Open Nepal: As global trends towards transparency and accountability gain ever-increasing traction, the phrase ‘open data’ is becoming increasingly used - both internationally and within Nepal.  But, what is Open Data and what exactly does it mean for Nepal?  This document answers some basic questions, and dispels some myths, about the increasing demands and challenges of ensuring ‘open data’.

Please read the Nepali version here.
Please read the English version here.

Briefing: Open Data and Right to Information

Freedom Forum: This briefing paper discusses the relation between Right to Information and Open Data, highlighting how open data can play a significant role in supporting Right to Information in Nepal.

Please read here.

Briefing: Open Government Data

Open Nepal: Open Government Data holds great potential to improve the effectiveness of development efforts as, with better access to government data, development actors can target their programs more effectively to the issues and communities that need most support. This briefing explains what open government data is, looks at how it could support Nepal’s development, and highlights the links with citizens’ legal right to information.

Please read the Nepali version here.
Please read the English version here.

Briefing: Open Government Partnership

Open Nepal: Around the world governments are taking important steps to promote more open, accountable and responsive governance. A global initiative that is supporting this trend is called the 'Open Government Partnership' (OGP). Nepal is not currently a member of the OGP but is eligible to join. This briefing presents the case for Nepal to join the OGP and commit to the principals of open government.

Please read the Nepali version here.
Please read the English version here.

Case-Studies: Profiles of Data Use in Nepal

Development Initiatives: To ensure that data sharing efforts respond to needs these series of profiles present personal stories of data use. The profiles aim to amplify the voices of those whom the data revolution seeks to support by showing how development data can have optimum impact on the ground.

Guide: Open Data Curriculum

Open Knowledge Nepal: The Open Data Curriculum is a course outline, with content and lessons to help conduct a course on open data at universities. It is accompanied by the Open Data Manual (see below).

Please read here: http://odap.oknp.org

Guide: Open Data Handbook (Nepali)

Open Knowledge Nepal: The handbook provides basic information about open data and its technicalities in simple way.  

Please read here: http://handbook.oknp.org

Guide: Open Data Manual

Open Knowledge Nepal: The Open Data Manual accompanies the Open Data Curriculum, and provides information on the basics of open data, with a list of helpful resources for the readers to refer to.

Please read here: http://odap.oknp.org

Report: Aid Data Needs and Use cases in Nepal

Data for Development Program: This research paper highlights and investigates the aid information needs of different stakeholders in order to assist user-centered approaches to providing data on aid finance in Nepal.

Please read here: http://devinit.org/post/aid-data-needs-use-cases-nepal/

Report: Exploring the Emerging Impacts of Open Aid Data and Budget in Nepal

Freedom Forum: The study discusses the context of open data on aid and budgets in Nepal, and the emerging use and outcomes from open data.

Please read here: http://www.opendataresearch.org/content/2014/724/exploring-emerging-imp…

Report: Nepal: Open Government Partnership Readiness

Accountability Lab: This report presents and analyses findings from the study conducted by Accountability Lab on the readiness of Nepal for the Open Government Partnership (OGP). It also highlights the challenges, and the next steps for Nepal to join the OGP.

Please read here: http://www.accountabilitylab.org/wp-content/uploads/2015/12/Nepal-OGP-R…

Report: Nepal’s Emerging Data Revolution

Development Initiatives: The paper shares information on Nepal’s emerging data revolution highlighting the role of government, civil society, private sector, and development partners in the data ecosystem in Nepal.

Please read here: http://devinit.org/post/insights-into-nepals-emerging-data-revolution/

Report: Open Budget (Nepali)

Freedom Forum: This study translates the technicalities of Nepal’s budget in a simplified form, educating the readers regarding the various components of a budget, from it formulation, to laws and policies governing budget decisions, and so on.

Please read here: http://freedomforum.org.np/content/download/open-budget/

Video: Open Data: A journey of discovery in Nepal

Development Initiatives, Publish What You Fund: This short documentary follows a Nepali journalist as he seeks open data to help him make a better future for his son.

Please view here: https://www.youtube.com/watch?v=gK6CF2K29j4

Video: Open Data Day Nepal Videos

Open Nepal and Young Innovations: This series of videos highlights events that have taken place over the past few years to celebrate open data day.

Video: Video-series busting development myths in Nepal

Bikas Udhyami: These seven short videos demystify different aspects of Nepal's development by explaining them on the basis of available data in an easy to understand manner.