USING GENERATIVE AI TO FORECAST ELECTRICITY DEMAND AND SUPPLY GAPS: SUB-SAHARAN AFRICA VS. OTHER REGIONS
Arthur Uzoma, Somi Fredrick, Abosede Omishade, Nina Garmash, Adeniyi Olaolu, Nishant Katiyar, Bharat Kumar Jhawar, Emmanuel Alafaa, Tolulope Oke, Ademo Vitalis, Abiodun Onifade, Oluchukwu Okorie
1. ABSTRACT
This research paper explores the application of generative AI to forecast electricity demand and supply gaps, focusing on Sub-Saharan Africa (SSA) compared to other regions. Despite efforts to improve access, around 600 million people in SSA still lack electricity, hindering the achievement of Sustainable Development Goal 7. By leveraging generative AI models, this study aims to predict electrification needs and provide actionable insights for better planning and decision-making. The paper also compares SSA’s electrification challenges with other regions to identify best practices and transferable strategies to improve electricity access in that region.
2. INTRODUCTION
Access to electricity is fundamental for economic growth and social development. However, in Sub Saharan Africa (SSA), approximately 600 million people still lack electricity, severely hindering the achievement of Sustainable Development Goal (SDG) 7, which aims for universal access to modern energy by 2030 (African Union Commission, 2021; International Energy Agency, 2023). This shortage impacts education, healthcare, business operations, and overall quality of life (World Health Organization, 2022). Innovative approaches are needed to effectively predict and manage electricity demand and supply gaps. The 2023 UN report, “Tracking SDG 7: The Energy Progress Report,” highlights that COVID-19 slowed electricity access in SSA, and the situation remains dire due to rising energy prices since mid-2021. In 2021, 675 million people lacked electricity access, with 567 million of these individuals residing in SSA, accounting for over 80% of the global population without electricity.
To tackle this, the project aims to develop a generative AI model to predict electrification demand and supply gaps in SSA, comparing these insights with other regions. This model will take into account various factors such as income levels, regional disparities, access to alternative energy sources, balance values, and electricity rates. By leveraging complex data patterns, the model aims to provide actionable insights for improved planning and decision-making (World Bank, 2020; United Nations, 2023).
3. LITERATURE REVIEW
Electricity Access in Sub-Saharan Africa: The issue of electricity access in Sub-Saharan Africa (SSA) has been thoroughly documented. The African Union Commission’s “Agenda 2063: The Africa We Want” outlines a vision for sustainable development, emphasizing universal energy access as essential for economic growth. Reports from the International Energy Agency (IEA) and the United Nations in 2023 highlight the slow progress in electrification efforts, revealing significant regional disparities. The World Bank’s 2020 report, “Barriers to Urban Electrification in Sub-Saharan Africa,” discusses various challenges including infrastructural deficits, financial constraints, and policy inefficiencies that hinder urban electrification. Additionally, the World Health Organization’s 2022 fact sheet on “Access to Electricity in Sub-Saharan Africa” provides insights into the health and socio-economic impacts of electricity access deficits.
The IEA’s “Africa Energy Outlook 2022” and “ World Energy Outlook 2021” offer comprehensive analyses of the energy landscape, projecting future trends and identifying critical areas for intervention. Bruegel’s 2021 report explores the role of international institutions in supporting electrification efforts in SSA, emphasizing the importance of coordinated global efforts. Research published in Science Direct underscores the correlation between electricity access and economic growth in SSA. McCoy, Cohen, and Lunt’s 2020 study, “Forecasting Electrification Needs in Africa: A Multiscale Approach,” highlights the potential of integrating macroeconomic trends with local-level data to create detailed demand projections.
These studies collectively illustrate the complexity of electrification challenges in SSA and the need for advanced predictive models.
Leveraging generative AI, this research aims to build on existing work and provide a robust tool for forecasting electricity demand and supply gaps, ultimately contributing to more effective and sustainable energy solutions for SSA.
Current State of Electricity Production: OECD vs. Global Trends: According to IEAdata for March 2024, OECD countries saw a slight decline (- 0.3%) in net electricity production compared to the previous year. Fossil fuels and renewables played critical roles, with fossil fuels accounting for 44.6% of the electricity mix, despite a decline in coal-fired generation. Renewables, particularly solar and hydro, experienced significant growth, making up 38.5% of the mix, with wind energy showing varied regional trends (IEA, 2024).
Energy Landscape in Sub-Saharan Africa: In contrast, Sub-Saharan Africa’s energy landscape differs significantly. The region accounts for only 3% of global electricity production, with per capita consumption at 0.636 MWh, much lower than the global average. Electricity generation in SSA relies heavily on natural gas (42%) and coal (28%), with limited penetration of renewable sources like hydro (77% of renewable generation) (IEA, 2021).
Challenges in Sub-Saharan Africa: The electrification rate in SSA remains low, with over 600 million people lacking access to electricity and nearly 1 billion without clean cooking supplies. The region’s energy infrastructure faces numerous challenges, including infrastructural deficits, financial constraints, and policy inefficiencies, which hinder efforts to improve electrification rates (IEA, 2024).
Role of Generative AI in Forecasting Electricity Demand: Generative AI offers promising solutions for forecasting electricity demand and supply gaps by leveraging historical data and demographic factors. By simulating various electrification scenarios and population growth patterns, generative models can provide insights into future trends, aiding in proactive planning and resource allocation (Hamoye, 2024).
Case Studies and Predictive Models: Several studies highlight the effectiveness of generative AI in predicting future electrification needs. Projects utilizing datasets from the World Bank and other sources have demonstrated the utility of AI technologies in refining analysis and identifying underlying factors influencing electrification demands (Kubeflow group, Open AI group, Hamoye, 2024).
This literature underscores the critical role of generative AI in addressing electricity access challenges in SSA by providing robust predictive models for electricity demand and supply. This technology can significantly contribute to more effective and sustainable energy solutions, bridging the electrification gaps in SSA and supporting its socio-economic development.
4. METHODOLOGY
This study employs a comprehensive analysis using Python and Tableau to identify trends, recognize patterns, variations, and potential disparities in electrification. Our approach involves detailed data collection, cleaning, and exploration methods to ensure the accuracy and reliability of our model. This section outlines our data preprocessing steps, the modeling and evaluation techniques used to develop a generative AI model for predicting electrification demand and supply gaps in Sub-Saharan Africa, compared to other regions.
4.1 Data Collection
For this research, we explored and gathered data from the following sources:
• Global Electrification Database by Harvard Dataverse which provides comprehensive data on electrification rates across various regions and countries.
• World Bank data that shows the percentage of the population with access to electricity by country.
• World Energy Statistics and Balances published by the International Energy Agency (IEA 50) that covers energy supply, transformation, and consumption across various fuels and sectors.
However, the Global Electrification Database provides data for only 124 countries up until 2015, and the IAE 50 data covers only OECD countries, excluding SSA countries which makes it irrelevant to our project. Therefore, it was necessary to source additional data.
To gather sufficient data for reliable results and a more accurate Generative AI model, we sourced additional data from the sources mentioned below:
• TRACKING SDG 7 — The Energy Progress Report which offers detailed statistics and analysis on various aspects of energy access, efficiency, and renewable energy deployment across different regions and countries.
• Our World in Data offers valuable insights into global energy access disparities.
• Ember data source provides comprehensive yearly data on global electricity generation by fuel type.
• UN Data Retrieval System is a comprehensive resource provided by the United Nations, offering a wide range of statistical data on various topics, including electricity access.
• Africa Energy Portal (AEP) is a comprehensive database providing detailed information and data on energy trends, policies, and projects across Africa.
4.2 Data Cleaning and Processing
Our data cleaning and processing began with a thorough examination to ensure accuracy and consistency. We verified and corrected data types, ensuring each column had the appropriate type of data. Categorical variables were encoded into numerical values or suitably transformed. Extensive null values were addressed by removing empty columns, leaving us with data from 1990- 2022, while other missing values were filled using methods such as forward fill, backward fill, KNNImputer, interpolation and prophet. For example, missing ‘net_imports’ values were calculated where both ‘el_imports’ and ‘el_exports’ were present. Most of the missing values were before 2000 and in 2023, so we focused on the 2000–2022 subset. South Sudan’s incomplete data led to the decision to drop its rows, while South Africa’s missing ‘renewables_other’ values were replaced with the mean for upper-middle-income countries.
Units of measurement were standardized for uniformity, and irrelevant data, columns, and rows were dropped.
We corrected spelling errors in country names to ensure accurate data merging.
The dataset was reorganized by pivoting it to place indicators into distinct columns, and datasets were merged on their common columns, creating a comprehensive dataset ready for analysis.
Several outliers were identified, notably South Africa and Seychelles. South Africa emerged as an outlier for numerous indicators, while Seychelles, the only high-income country in the dataset, also stood out. These outliers are genuine; their values are accurate representations of their respective contexts and cannot be disregarded or adjusted. The presence of these outliers underscores the significant disparities between countries within Sub-Saharan Africa (SSA). Consequently, our modeling approach must account for these discrepancies. Therefore, we require a model that is robust to outliers, ensuring that these anomalies do not skew the overall analysis but rather highlight the existing imbalances within the region.
This comprehensive approach ensured the quality and integrity of our Dataset, readying it for further analysis.
4.3 Data Exploration and Visualization Our exploratory analysis aims to uncover the discrepancies between electricity demand and supply in Sub-Saharan Africa compared to other regions. This involves identifying gaps where the electricity needs are not being met and comparing these findings to other parts of the world. By doing so, we can better understand the unique challenges faced by Sub-Saharan Africa in terms of electricity access and reliability.
How does the installed electricity capacity vary between Sub-Saharan Africa countries?
Figure 1: Comparison of Installed Electricity Capacity between Sub Saharan Africa and Other Regions
Figure 1 compares the installed electricity capacity across various countries in Sub-Saharan Africa. South Africa has the highest installed electricity capacity, significantly surpassing all other countries, with Nigeria and Angola following at a distance. Most other countries have relatively low installed capacities, with many displaying capacities below 2000 MW. This highlights the disparity in electricity infrastructure within the region, with a few countries having substantial capacity while the majority lag far behind.
How has the overall electricity access evolved over time in Sub-Saharan Africa compared to other regions?
Figure 2: Total Electricity Access Over The Years for All Countries
Figure 2 is a plot for different regions that shows trends in total electricity access. Africa and Sub Saharan Africa show a significant increase in electricity access starting from approximately 30% in the early 1990s to about 50–60% in recent years compared to other regions like Asia, East Asia and Pacific (WB), Europe, Europe and Central Asia (WB), Latin America and Caribbean, and Latin America and Caribbean (WB) which all show high levels of electricity access, consistently close to or above 90% throughout the years. There are sharp declines and recoveries in certain regions, such as a noticeable dip in the yellow line representing the World. The global trend indicates improvement, some regions still lag significantly behind others in achieving universal electricity access.
What is the relationship between GDP per capita and electricity access in different regions?
Figure 3: Relationship between GDP per Capita and Electricity Access
Figure 3, illustrates the relationship between GDP per capita and electricity access (%). There is a clear positive correlation, indicating that countries with higher GDP per capita tend to have higher electricity access. Most of the data points cluster at lower GDP per capita values, with a wide range of electricity access, but as GDP per capita increases beyond approximately $5000, electricity access generally approaches 100%. This suggests that economic development is strongly linked to better electricity infrastructure and access.
What is the composition of electricity generation sources in Sub-Saharan Africa compared to other regions?
Figure 4: Percentage Breakdown of Electricity Supply Sources by Region
Figure 4, illustrates the percentage breakdown of electricity supply sources by region. Fossil fuels (coal, oil, and gas) dominate the energy supply in many regions, especially in Africa, where they account for a substantial portion. In Europe, nuclear energy makes a significant contribution to the electricity supply, alongside a notable share of renewables like wind and solar. Regions such as Latin America and the Caribbean and East Asia and Pacific show a more diversified energy mix with considerable contributions from hydroelectric power. Figure 4 highlights the varying dependency on different energy sources across regions, emphasizing the significant role of fossil fuels in some areas and the growing importance of renewables in others.
What trends can be observed in the adoption of renewable energy sources across various regions?
Figure 5: Renewable vs Non-Renewable Energy Supply for All Regions
Figure 5, compares the supply of renewable and non-renewable energy across various regions. It shows that the supply of renewable energy is significantly lower than non-renewable energy in each region. Notably, Africa’s renewable energy supply is minimal compared to its non-renewable supply. In contrast, regions such as Europe and Asia exhibit a higher supply of non-renewable energy, with Asia surpassing 200,000 units. This highlights the substantial reliance on non renewable energy sources across the globe, with renewable energy playing a much smaller role in the overall energy supply.
How does the population without electricity access vary across regions over time?
Figure 6: Population without Electricity Access Across Regions Over Time
Figure 6 indicates that the percentage of the population without electricity access is rising over time in all the displayed regions. For instance, in Angola, the percentage increased from approximately 2% in 2000 to around 8% in 2015. The top five regions with the highest population lacking electricity access are Nigeria, Ethiopia, the Democratic Republic of Congo, Tanzania, and Uganda. The data shows that some countries, such as Nigeria and Ethiopia, consistently have a high percentage of their population without electricity access, while others, like South Africa, have significantly lower percentages, indicating better electricity access. The trend lines for many countries appear relatively stable, with minor fluctuations over the years, suggesting that electricity access issues remain persistent in certain regions. This highlights the ongoing challenge of improving electricity access in many parts of Africa.
4.4 Data Modeling, Evaluation and Deployment
4.4.1 Feature Engineering
In our feature engineering process, we initially reviewed 39 columns from the original Dataset. We excluded 12 columns that were either less important or highly correlated, focusing on the most relevant indicators for future modeling. The primary objective of our project is to analyze the ‘electricity demand and supply gap’, defined as the difference between the required electricity and the actual electricity delivered to the population. To achieve this, we developed new metrics to measure the gap effectively. We considered the following categories:
SUPPLY: Represented by ‘el_demand’, it refers to the electricity delivered (supplied) within a country, including both generated and imported electricity, measured in TWh.
ACCESS: Denoted by ‘el_access_total’, it indicates the percentage of the total population with access to electricity
SUPPLY RATE(supply_rate): Calculated as SUPPLY / ACCESS, represented by ‘el_demand’ / ‘el_access_total’, it measures the electricity delivered to 1% of the population with access to electricity, measured in TWh/%.
TARGET DEMAND(t_demand): Defined as 100% * SUPPLY RATE, it estimates the electricity needed to provide access to 100% of the population, calculated as 100% * ‘ e l _ d e m a n d ‘ / ‘el_access_total’, measured in TWh.
GAP(gap): Calculated as TARGET DEMAND — SUPPLY, it represents the missing amount of electricity, indicating the difference between the needed and the delivered electricity, measured in TWh. The new metrics calculated, ‘supply_rate’, ‘t_demand’, and ‘gap‘ , provide a comprehensive understanding of the electricity demand and supply dynamics in the regions studied. These metrics were derived based on the formulas mentioned above, ensuring a robust approach to measure the electricity gap accurately.
4.4.2 Modeling
This section examines the application of AI models to identify trends and gaps in electricity demand and supply between Sub-Saharan Africa (SSA) and other regions. The chosen models are the Gradient Boosting Regressor, Sequential with Dense and Sequential with Long Short Term Memory (LSTM), selected for their robustness and ability to detect complex patterns in data, leading to accurate results through continuous performance improvement.
• Gradient Boosting Regressor: This model is leveraged for its high predictive performance, effectively identifying gaps in electrification demand and supply.
• Keras Sequential with Dense Model: This fully connected neural network is capable of learning increasingly complex hierarchical representations of data.
• Long Short Term Memory (LSTM): LSTM is ideal for this task due to its ability to manage long-term patterns and dependencies in the data.
4.4.3 Evaluation
To assess the performance of the models on the provided data, three metrics were employed: Mean Absolute Error (MAE), Mean Squared Error (MSE), and Root Mean Squared Error (RMSE), with RMSE being the primary metric used.
Gradient Boosting Regressor: The Gradient Boosting Regressor was evaluated using both RMSE and MAE on the training and test sets across different numbers of estimators (figure 7 and 8). This process determined the number of iterations required for the model to achieve convergence. Both metrics converged at a local minimum after 20 estimations for both the training and test sets, with final scores of 1.24 for MAE and 2.93 for RMSE. The MSE produced a score of 8.56. Overall, the model performed well, correctly predicting the gaps in electrification demand and supply across countries.
Figure 7: MAE over Number of Estimators(L) RMSE over Number of Estimators(R)
Figure 8: Actual vs Predicted Values
Keras Dense Sequential Model: The Keras Dense Sequential model was initialized with 64 units and a ReLU activation function (Figure 9 and 10). A dropout of 0.5 was added after each layer to prevent the model from overly relying on specific features. The final output layer was a dense layer with a single unit. The model’s performance was evaluated using RMSE, resulting in an overall score of 5.05. Figure 10, shows the training and validation loss plotted against epochs.
The model learned the training data well and predicted the correct outputs, though performance fluctuated after 10 epochs.
Figure 9: Actual vs Predicted Values
Figure 10: Training and Validation Loss
Long Short Term Memory (LSTM): The LSTM model, a generative model, was trained (Figure 11 and 12) iteratively on the training and test data. The first layer was initialized with 50 units and ReLU activation. The model achieved a training loss of 21.08 and a test loss of 15.65, performing better on the test set than the average. The overall MSE score was 4.41. This iterative training helped refine the model’s ability to handle long-term dependencies and patterns, ensuring accurate predictions of gaps in electricity demand and supply across the regions under study.
Figure 11: Training and Validation Loss
Figure 12: Actual vs Predicted Values
4.4.4 Deployment
We successfully deployed our machine learning model using Flask and Render to create a seamless and interactive user experience. Initially, we meticulously prepared and cleaned the data, selecting key features for effective model training. Our neural network model was then trained and evaluated to ensure high accuracy and reliability.
The deployment process involved several key components:
• Backend Engine: Flask serves as the backend engine, handling user requests, processing data, and returning predictions.
• User Interface: Built with HTML, the user interface provides a user-friendly way for individuals to input data and view predictions.
• Deployment Platform: Render simplifies the deployment process by linking our project repository, configuring the service, and making the application live.
During the live demonstration, we showcased the application’s functionality and user experience, emphasizing its accessibility, scalability, and maintainability. Thorough testing and documentation ensure the application’s reliability and readiness for real-world use. Our streamlined deployment process guarantees a robust and user-friendly application that meets stakeholders’ needs, with plans for ongoing monitoring and improvement based on user feedback.
5. RESULT
The Gradient Boosting Regressor demonstrated superior performance in predicting the gaps in electrification demand and supply, achieving the lowest error metrics among the models evaluated. These low error metrics indicate that the Gradient Boosting Regressor can reliably predict discrepancies between electrification demand and supply across various countries, making it a valuable tool for planning and optimizing electricity distribution.
Keras’ Dense Sequential model, although not as precise as the Gradient Boosting Regressor, still performed well. The inclusion of dropout layers effectively prevented overfitting, ensuring that the model did not overly rely on specific features and could generalize better to unseen data. The model showed minimal variation between the actual and predicted gaps, indicating reliable performance.
The LSTM model, designed to handle sequential data and long-term dependencies, also showed promising results. With a training loss of 21.08 and a test loss of 15.65, the model demonstrated effective learning and generalization capabilities. However, the absolute values of the training and test losses suggest that there remains a gap between the model’s predictions and the actual values. This discrepancy indicates that while the LSTM model is more accurate than previous models, there is still room for improvement to achieve more precise predictions.
Overall, each model exhibited a strong ability to predict electrification demand and supply gaps. The Gradient Boosting Regressor excelled in both training and test set performance, making it the most effective model for this task. These models provide robust tools for accurately predicting electrification demand and supply gaps, aiding in effective planning and resource allocation across regions.
6. CONCLUSION
This study underscores the potential of generative AI models to offer accurate and actionable insights into electrification challenges. By leveraging these models, stakeholders can address gaps in electricity demand and supply, contributing to improved electrification efforts across SSA and beyond.
The re-evaluation process has significantly enhanced our model’s performance, with the optimized LSTM model now providing accurate predictions for electricity demand and supply gaps in SSA. These insights are crucial for informed energy planning and policy-making, ultimately aiding in improving energy access in different regions.
Our deployment process is streamlined and efficient, with a user-friendly web interface facilitating easy interaction with our model. The application is live, functional, and ready for use. Next course of action includes monitoring the application’s performance and gathering user feedback for further improvements and enhancements.
7. REFERENCES
1. African Union Commission. (2021). Agenda 2063: The Africa We Want.
https://au.int/en/agenda2063/overview.
2. Bruegel. (2021). Electrification in Sub Saharan Africa: The Role of International Institutions.
https://www.bruegel.org/report/electrification -sub-saharan-africa-role-international institutions
3. Ember. (2022). Yearly Full Release Long Format. https://ember
climate.org/app/uploads/2022/07/yearly_full _release_long_format.csv.
4. Hamoye. (2024). Premiere Projects and Capstone Reports. [Hamoye Premiere Projects](Hamoye Premiere Projects).
5. International Energy Agency. (2021). World Energy Outlook 2021.
https://www.iea.org/reports/world-energy outlook-2021
6. International Energy Agency. (2022). Africa Energy Outlook 2022.
https://www.iea.org/reports/africa-energy outlook-2022.
7. International Energy Agency. (2023). Tracking SDG 7: The Energy Progress Report 2023.
https://www.iea.org/reports/tracking-sdg7.
8. International Energy Agency (IEA). (2024). Monthly Electricity Statistics. [IEA Monthly Electricity Statistics](IEA Monthly Electricity Statistics).
9. McCoy, A. B., Cohen, J. L., & Lunt, P. B. (2020). Forecasting Electrification Needs in Africa: A Multiscale Approach. Global Environmental Change, 62, 102068.
doi:10.1016/j.gloenvcha.2020.102068.
10. Our World in Data. (n.d.). Share with Access to Electricity vs Per Capita Energy Consumption.
https://ourworldindata.org/grapher/share with-access-to-electricity-vs-per-capita energy-consumption.
11. Sciencedirect. (2022). Electricity Access and Economic Growth in Sub-Saharan Africa.
https://www.sciencedirect.com/science/ article/pii/S2468227622003209.
12. United Nations. (2023). Tracking SDG 7: The Energy Progress Report.
https://sdg7.org/reports/2023-energy progress-report.
13. World Bank. (2020). Barriers to Urban Electrification in Sub-Saharan Africa: A Perspective from End-Users.
https://blogs.worldbank.org/en/developmentt alk/barriers-urban-electrification-sub
saharan-africa-perspective-end-users.
14. World Bank Group. (2021). Energy Progress Report.
https://www.worldbank.org/en/topic/energy/p ublication/energy-progress-report.
15. World Health Organization. (2022). Access to Electricity in Sub-Saharan Africa.
https://www.who.int/news-room/fact
sheets/detail/electrification-in-africa.