Country Coding Considerations for Dataset Harmonization and Applied Uses

Introduction

Managing country codes is one of the most difficult tasks in environment-security analysis, and each dataset treats the concept differently. Country code designations may be very strict and only respect internationally recognized sovereign nations starting with their first official day of independence. Other datasets may include separate observations and codes for territories, disputed states, colonial nations, and other quasi-independent states. Coding systems like the Correlates of War (CorrelatesofWarProject2017?) or Gleditsch and Ward (Gleditsch1999?) fall into the former with strict, to the day, international recognition, while refugee and asylum datasets provided by the United Nations (UnitedNations2017?) falls into the latter. UN migration and refugee datasets include nearly every conceivable territorial designation/disaggregation (e.g. Puerto Rico, Taiwan, Faroe Islands, etc.).

When using an individual dataset, users need a general awareness for the conceptual approach to the country code specification and any potential inconsistencies, however, when attempting to harmonize multiple datasets, more intricate knowledge is required. Merging multiple datasets without first addressing disparate approaches to country coding will lead to inconsistencies, numerous NA values, and compound downstream error in the analysis. It takes time and experience to anticipate country coding issues and know what to be on the lookout for when undergoing project planning. The goal of this vignette is to outline common issues and considerations for harmonizing multiple country-year datasets. This walkthrough uses the Varieties of Democracy (V-Dem) (Coppedge2020?) and Correlates of War coding schemes to illustrate recurrent challenges. They are good datasets to work with while learning for several reasons:

• They represent opposite ends of the country-coding ideological spectrum.
  • Because V-Dem is focused on polities, it is highly disaggregated and observes long standing claims of independence.
  • Conversely, CoW is very strict and adheres to official and internationally recognized claims of independence.
• Both datasets are consistent and have little to no errors and inconsistencies. This is helpful for learning, but not entirely representative of real world experiences. On top of differences in coding ideologies, many datasets have a slew of errors and inconsistent applications that further complicate pre-processing.
• They both have direct programmatic access via R that makes management, exploration, and visualization easier.
  • The vdemdata package (V-DemInstitute2020?).
  • The states package (Beger2020?) featuring CoW and G&W code access and manipulation.
  • The cshapes package (Weidmann2010?; Weidmann2016?); vectorized historic state boundaries using either the CoW or G&W schemes.
• V-Dem provides reference materials that are extremely helpful for learners.
  • The Country Coding Manual and Historic Name column in the raw data provide a lot of context for the year to year regime changes, and their reasoning for important decisions.

V-Dem Country Codes

The Varieties of Democracy (V-Dem) presents a collection of highly disaggregated indicators (400+) depicting wide-ranging measures of democracy and institutional characteristics dating back to 1789. In contrast to several other political, social, and environmental datasets, V-Dem demonstrates an extreme level of transparency in their methods and copious documentation. This includes a dedicated supplementary manual depicting their country coding approach; V-Dem Country Coding Units v11.1 (Coppedge2021?). This tutorial will highlight the most relevant (in my opinion) portions of their approach and country-year coding decisions.

V-Dem defines a country as

…a political unit enjoying at least some degree of functional and/or formal sovereignty.

Generally speaking, V-Dem will have a country-year observation for a country if:

1. The state made a formal declaration of independence; even if not yet fully recognized by the international community.
2. In modern times this concerns states like Kosovo, Taiwan, and Western Sahel, in historic times this concerns states such as Colonial Asia and Africa.
3. If the state in question operates with some degree of autonomy that distinguishes itself, at least in its polities and institutions, from the parent nation.

In my limited experience with V-Dem, this conceptual approach is carried out very consistently, but additional pre-processing is required to successfully merge with more strict datasets. Moreover, the Country Coding Units Manual contains information that helps users construct historical time series of nations that have limited independent observations in the dataset. For example, Bangladesh is coded independently starting in 1971, but to construct a longer historic record, the Country Coding Units Manual states that the following combination of observations may be used:

1. India (Princely state of British India (1910 – 1947))
2. Pakistan (1947 - 1971)
3. Bangladesh (1971 - )

The Country Coding Units Manual is a great reference if you need to combine historical democracy indicators for nations with intermittent internationally recognized sovereignty with other historically disaggregated environmental, migration, or conflict data.

The histname variable in the raw V-Dem dataset is also an excellent source of context for the true nature of regime changes in annual polity. While the country_name field is static and typically reflects the modern, common, country name in the V-Dem dataset, the histname reflects official or historic names and potential states of occupation. Take Somalia for example:

V-Dem country name, V-Dem historic designation, Correlates of War country code, and start and end years for Somalia in the raw V-Dem Version 11.1 dataset.

country_name	histname	COWcode	Start	End
Somalia	Somalia under formal Italian control over most of the territory [except British Somaliland]	520	1900	1909
Somalia	Somalia under effective (more or less) Italian control	520	1910	1940
Somalia	Somalia under British occupation	520	1941	1949
Somalia	UN Trust Territory of Somalia under Italian administration	520	1950	1959
Somalia	Somali Republic [unites Somaliland with the Trust Territory of Somalia]	520	1960	1968
Somalia	Somali Democratic Republic	520	1969	1990
Somalia	Somali Democratic Republic [civil war]	520	1991	2003
Somalia	Transitional Federal Government [civil war]	520	2004	2011
Somalia	Federal Republic of Somalia [civil war]	520	2012	2021

This provides a clear understanding of Somalia’s historical sovereignty and polity for the past 120 years. When combined with the Country Coding Units Manual, the user is able to make more informative choices regarding their research interests. It also serves as excellent reference material for historical nation-states that is much faster than being sucked into a Wikipedia black hole.

Now we will walk through some of the more important country coding considerations for applied cases using Varieties of Democracy and Correlates of War for 1900-2020.

The Americas

Europe

Middle East and North Africa

Non-MENA Africa

There are not many complicated distinctions throughout Africa. V-Dem codes most African nations independently starting in 1900 while indicating their colonial protectorate. Depending on the colonial parent nation, most African countries acquired internationally recognized independence between 1955-1975; this is when the CoW record begins. Some notable African countries:

• Somaliland
  • Recognized by V-Dem as an independent state from 1900-1960 and again starting in 1991-.
  • CoW (and several other datasets) does not include Somaliland.
• Ethiopia
  • Coded continuously by V-Dem starting in 1789; including Italian occupation from 1936-1941.
  • Coded intermittently by CoW (1898-1936; 1941-); excludes Italian occupation.
• Sudan and South Sudan
  • They are coded identically by V-Dem and CoW; a separate code for South Sudan starting in 2011.

Asia

V-Dem includes a greater historical record for most Asian countries, because, similar to Africa and the Caribbean, there was less centralized control exercised over their polities and institutions during colonial rule. As a result, most Asian states are recorded in V-Dem beginning sometime 1789-1900, and in CoW during one of the waves of colonial (1945-1950) or regional (1971-1975) independence. However, there are a few nation-states with more complicated records in V-Dem and CoW:

References