AI-driven Predictive Model for Resettlement Outcomes: Using artificial intelligence for resettlement monitoring and post-evaluation

Country: Mongolia

Start of implementation: 2022

Technology: Artificial intelligence

ITD Partner ADB Department: East Asia Department

ADB Partners:

·       Mobiva—technology service provider

·       Government of Mongolia

·       Project Management Office of the Mongolia Ulaanbaatar Urban Services and Ger Areas Development Investment Program

Field visits and desk reviews help ensure that land acquisition and resettlement (LAR) impacts and risks of ADB-supported projects are identified. Appropriate measures can then be developed and implemented to avoid or otherwise minimize, mitigate, and compensate for any and all adverse impacts.

Data-driven decision-making is necessary for resettlement planning and implementation. However, there were no models in place to forecast positive or negative outcomes of resettlement projects. In addition, there was no system in place to compare and analyze outcomes across projects. Inaccurate or delayed assessments due to data issues can affect compliance with safeguards requirements. Delays could also potentially negatively impact communities affected by projects.

This initiative saw the development of an AI-driven platform to carry out real-time resettlement monitoring and predict potential outcomes in Mongolia. Specifically, it tested whether machine learning could be used to predict resettlement outcomes using a combination of variables, including project land and acquisition requirements and baseline socioeconomic conditions of affected households.

This was the second phase of the testing of the platform that was developed by Mobiva, the technology service provider selected in ADB’s “Real-time Tracking of Resettlement Implementation” challenge. The team was originally tasked with developing a proof of concept (POC) of a dashboard that allowed users to collect and share real-time resettlement data via a digital platform. The POC also illustrated how it could be used to visualize resettlement information.

The initiative, which was done under the Mongolia Ulaanbaatar Urban Services and Ger Areas Development Investment Program Tranche 1 and Tranche 2, was the first time for the bank to test a machine learning model for post-resettlement evaluation. This solution was in line with ADB’s East Asia Department’s digital transformation agenda andthe Government of Mongolia’s E-Mongolia policy (Vision 2050).

The key activities were as follows:

Data matching and cleaning. The pre- and post-LAR datasets were first matched to ensure that the same households were compared. The Tranche 1 data resulted in 144 entries that were successfully matched. Entries with duplicate and null values were also removed as part of data cleaning.

Data pre-processing and normalization. The categorical data were converted into numerical data to make it suitable for machine learning algorithms. Data normalization was then done to mitigate any bias due to scale disparities. A histogram based on the available data was also created to show the changes in monthly household income from pre-LAR to post-LAR.

Model design and training. Different AI models were developed and tested based on the Tranche 1 household data: decision tree, artificial neural networks, and k-nearest neighbor. Testing showed that the artificial neural network model was better at making predictions than k-nearest neighbor and decision tree, particularly for the “No change” and “Increase” categories. Three AI models were developed: household income, household poverty, and household satisfaction. The performance of these models in forecasting resettlement outcomes was tested using Tranche 2 data.

The Tranche 2 datasets were also pre-processed and cleaned to ensure data quality and consistency. The clean Tranche 2 data were used to test whether the model was able to accurately forecast resettlement outcomes. The F-1 score, which measured precision and recall, was used to assess the models’ performance. The results were as follows:

• Household income. The model showed significant improvements in precision, recall, and accuracy, indicating the robustness in the model’s ability to predict changes in household income.

• Household poverty. While recall improved in Tranche 2, both accuracy and precision dropped in Tranche 2, which may be due to the higher rate of false positives.

• Household satisfaction. Precision, recall, and accuracy all dropped in Tranche 2, raising questions about the model’s ability to capture nuanced factors that affect household satisfaction. Another possible reason for the significant decrease across all indicators was the replacement of certain terms, which may have affected the model’s forecasting ability.

The models for predicting changes in household income and household poverty have the potential to be used in future resettlement projects, subject to additional training and testing using larger datasets to improve accuracy.

The model used to predict household satisfaction, on the other hand, may need to be revisited considering

that the Tranche 2 pre- and post-LAR datasets changed. Additional testing may be done to see whether the results would still be the same if the pre- and post-LAR questions remained consistent.

The models can be trained and tested using the same datasets consistently from the start to the end of the resettlement process.