Week 3

After we built the sample app that combined database, DBT, Svelte, and D3, we are about the build the actual website to display the index insurance analysis. The team members who joined the project last year already built a website for Zambia, and they are currently trying to add more components and analytical results to it, and reducing the data loading time as it could take a very long time to load and analyze the data. Professor Wu suggested us to build a new app for a different country, Ethiopia, where the Financial Instruments Sector Team used different parameters and calculations to calculate the payout rate, resulting different DBT models and analysis.

We started with gathering the raw data and reading the R code Financial Instruments Sector Team wrote that analyze the payout. We then compared the data with the one for Zambia. Then we spent a lot of time trying to understand the DBT models our team members wrote for Zambia data, and the R codes .After that, we also tried to analyze the differences of them for different countries. One of the differences is that the agricultural season in southern Africa crosses a calendar year, and it is not acrossing a calendar year in Ethiopia or west Africa. Some of the raw data files we need to use, and some of the intermediate calculations are also different for each country.

After we got the general understanding about the data and calculations, we started to write our own DBT models. We first realized that the raw data structure is different between Zambia and Ethiopia, so we wrote some preprossing Python script to convert the structure similar to the ones for Zambia. After spent a bit of time writing the models, finally, we were able to generate the combined payout, each individual payout, and capped climatology and have them saved as tables in our postgreSQL database.

Written on June 6, 2022