Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

📊 migration flow mdims #4135

Merged
merged 35 commits into from
Mar 26, 2025
Merged

📊 migration flow mdims #4135

merged 35 commits into from
Mar 26, 2025

Conversation

antea04
Copy link
Contributor

@antea04 antea04 commented Mar 18, 2025

PR to update the migration flows explorer (https://github.com/owid/owid-issues/issues/1734). Using the new methods, we (in tandem with lucas) have created a mdim as well as an explorer for this indicator.

Multidim on staging

Explorer on staging


Things to ask lucas about when he's back (does not affect merging):

  • Editing the mdim/ explorer config per view required copying the dictionary object first. Otherwise each change would affect all the views. Is this expected/ desired behaviour?

@owidbot
Copy link
Contributor

owidbot commented Mar 18, 2025

Quick links (staging server):

Site Dev Site Preview Admin Wizard Docs

Login: ssh owid@staging-site-data-migration-flow-mdims

chart-diff: ✅ No charts for review.
data-diff: ❌ Found differences
= Dataset garden/cdc/latest/measles_cases
  = Table measles_cases
    ~ Column cases (changed metadata, changed data)
-       -     date_accessed: '2025-03-25'
        ?                              ^
+       +     date_accessed: '2025-03-21'
        ?                              ^
-       -     date_published: '2025-03-20'
        ?                              ^^
+       +     date_published: '2025-03-13'
        ?                              ^^

        ~ Changed values: 1 / 41 (2.44%)
                country  year  cases -  cases +
          United States  2025      378      301
    ~ Column outbreaks_n (changed metadata)
-       -     date_accessed: '2025-03-25'
        ?                              ^
+       +     date_accessed: '2025-03-21'
        ?                              ^
-       -     date_published: '2025-03-20'
        ?                              ^^
+       +     date_published: '2025-03-13'
        ?                              ^^
    ~ Column states_with_cases (changed metadata, changed data)
-       -     date_accessed: '2025-03-25'
        ?                              ^
+       +     date_accessed: '2025-03-21'
        ?                              ^
-       -     date_published: '2025-03-20'
        ?                              ^^
+       +     date_published: '2025-03-13'
        ?                              ^^

        ~ Changed values: 1 / 41 (2.44%)
                country  year  states_with_cases -  states_with_cases +
          United States  2025                   18                   15
= Dataset garden/climate/2025-01-07/surface_temperature
  = Table surface_temperature
    ~ Dim country
-       - Removed values: 18396 / 199290 (9.23%)
                time                                          country
          2020-05-15                             Arctic Ocean (NIAID)
          2020-06-15                                Australia (NIAID)
          1947-05-15                     Mediterranean Region (NIAID)
          2024-06-15                     Mediterranean Region (NIAID)
          1943-08-15 South China and Easter Archipelagic Seas (NIAID)
    ~ Dim time
-       - Removed values: 18396 / 199290 (9.23%)
                                                   country       time
                                      Arctic Ocean (NIAID) 2020-05-15
                                         Australia (NIAID) 2020-06-15
                              Mediterranean Region (NIAID) 1947-05-15
                              Mediterranean Region (NIAID) 2024-06-15
          South China and Easter Archipelagic Seas (NIAID) 1943-08-15
    ~ Column anomaly_above_0 (changed metadata, changed data)
-       -   The difference of a specific month's average surface temperature when it exceeds the historical average from the 1991-2020 mean, in degrees Celsius.
        ?         ^^^^^ --
+       +   The deviation of a specific month's average surface temperature when it exceeds the historical average from the 1991-2020 mean, in degrees Celsius.
        ?        ++ ^^^^

-       - Removed values: 18396 / 199290 (9.23%)
                                                   country       time  anomaly_above_0
                                      Arctic Ocean (NIAID) 2020-05-15         1.062099
                                         Australia (NIAID) 2020-06-15         0.629733
                              Mediterranean Region (NIAID) 1947-05-15         0.003561
                              Mediterranean Region (NIAID) 2024-06-15         1.566858
          South China and Easter Archipelagic Seas (NIAID) 1943-08-15             <NA>
    ~ Column anomaly_below_0 (changed metadata, changed data)
-       -   The difference of a specific month's average surface temperature when it falls below the historical average from the 1991-2020 mean, in degrees Celsius.
        ?         ^^^^^ --
+       +   The deviation of a specific month's average surface temperature when it falls below the historical average from the 1991-2020 mean, in degrees Celsius.
        ?        ++ ^^^^

-       - Removed values: 18396 / 199290 (9.23%)
                                                   country       time  anomaly_below_0
                                      Arctic Ocean (NIAID) 2020-05-15             <NA>
                                         Australia (NIAID) 2020-06-15             <NA>
                              Mediterranean Region (NIAID) 1947-05-15             <NA>
                              Mediterranean Region (NIAID) 2024-06-15             <NA>
          South China and Easter Archipelagic Seas (NIAID) 1943-08-15        -0.870668
    ~ Column temperature_2m (changed data)
-       - Removed values: 18396 / 199290 (9.23%)
                                                   country       time  temperature_2m
                                      Arctic Ocean (NIAID) 2020-05-15       -2.810479
                                         Australia (NIAID) 2020-06-15       15.842676
                              Mediterranean Region (NIAID) 1947-05-15       18.343164
                              Mediterranean Region (NIAID) 2024-06-15       23.736559
          South China and Easter Archipelagic Seas (NIAID) 1943-08-15        26.36948
    ~ Column temperature_anomaly (changed metadata, changed data)
-       - description_short: The difference of a specific month's average surface temperature from the 1991-2020 mean, in degrees Celsius.
        ?                          ^^^^^ --
+       + description_short: The deviation of a specific month's average surface temperature from the 1991-2020 mean, in degrees Celsius.
        ?                         ++ ^^^^

-       - Removed values: 18396 / 199290 (9.23%)
                                                   country       time  temperature_anomaly
                                      Arctic Ocean (NIAID) 2020-05-15             1.062099
                                         Australia (NIAID) 2020-06-15             0.629733
                              Mediterranean Region (NIAID) 1947-05-15             0.003561
                              Mediterranean Region (NIAID) 2024-06-15             1.566858
          South China and Easter Archipelagic Seas (NIAID) 1943-08-15            -0.870668
= Dataset garden/education/2023-07-17/education_barro_lee_projections
  = Table education_barro_lee_projections
    ~ Dim country
+       + New values: 33 / 12248 (0.27%)
           year country
           1830 Oceania
           1875 Oceania
           1905 Oceania
           1920 Oceania
           1940 Oceania
    ~ Dim year
+       + New values: 33 / 12248 (0.27%)
          country  year
          Oceania  1830
          Oceania  1875
          Oceania  1905
          Oceania  1920
          Oceania  1940
    ~ Column f_adults__25_64_years__percentage_of_no_education (new data, changed data)
+       + New values: 33 / 12248 (0.27%)
          country  year  f_adults__25_64_years__percentage_of_no_education
          Oceania  1830                                               <NA>
          Oceania  1875                                          63.042763
          Oceania  1905                                          33.708057
          Oceania  1920                                          17.011372
          Oceania  1940                                           6.374139
        ~ Changed values: 35 / 12248 (0.29%)
          country  year  f_adults__25_64_years__percentage_of_no_education -  f_adults__25_64_years__percentage_of_no_education +
            World  1920                                            76.639046                                            76.396049
            World  1940                                            67.551224                                            67.276291
            World  1950                                              63.2355                                            62.930649
            World  1980                                            42.577347                                            42.365795
            World  1985                                            39.264984                                            39.072979
    ~ Column f_youth_and_adults__15_64_years__average_years_of_education (new data, changed data)
+       + New values: 33 / 12248 (0.27%)
          country  year  f_youth_and_adults__15_64_years__average_years_of_education
          Oceania  1830                                                         <NA>
          Oceania  1875                                                     1.783469
          Oceania  1905                                                     3.448642
          Oceania  1920                                                     4.643779
          Oceania  1940                                                     6.913009
        ~ Changed values: 35 / 12248 (0.29%)
          country  year  f_youth_and_adults__15_64_years__average_years_of_education -  f_youth_and_adults__15_64_years__average_years_of_education +
            World  1920                                                       1.440069                                                       1.453125
            World  1940                                                       2.287405                                                       2.308193
            World  1950                                                       2.629588                                                       2.657026
            World  1980                                                       4.685742                                                       4.719159
            World  1985                                                       5.157459                                                       5.188292
    ~ Column f_youth_and_adults__15_64_years__percentage_of_no_education (new data, changed data)
+       + New values: 33 / 12248 (0.27%)
          country  year  f_youth_and_adults__15_64_years__percentage_of_no_education
          Oceania  1830                                                         <NA>
          Oceania  1875                                                    59.347782
          Oceania  1905                                                    28.892849
          Oceania  1920                                                    14.470279
          Oceania  1940                                                     5.725431
        ~ Changed values: 35 / 12248 (0.29%)
          country  year  f_youth_and_adults__15_64_years__percentage_of_no_education -  f_youth_and_adults__15_64_years__percentage_of_no_education +
            World  1920                                                      73.855782                                                       73.61377
            World  1940                                                      60.772724                                                      60.525333
            World  1950                                                      56.415245                                                      56.143505
            World  1980                                                      36.597149                                                      36.414547
            World  1985                                                      33.601372                                                      33.436928
    ~ Column female_over_male_average_years_of_schooling (new data)
+       + New values: 33 / 12248 (0.27%)
          country  year  female_over_male_average_years_of_schooling
          Oceania  1830                                         <NA>
          Oceania  1875                                         <NA>
          Oceania  1905                                         <NA>
          Oceania  1920                                         <NA>
          Oceania  1940                                         <NA>
    ~ Column female_over_male_share_some_formal_education (new data)
+       + New values: 33 / 12248 (0.27%)
          country  year  female_over_male_share_some_formal_education
          Oceania  1830                                          <NA>
          Oceania  1875                                          <NA>
          Oceania  1905                                          <NA>
          Oceania  1920                                          <NA>
          Oceania  1940                                          <NA>
    ~ Column female_over_male_share_with_no_education (new data)
+       + New values: 33 / 12248 (0.27%)
          country  year  female_over_male_share_with_no_education
          Oceania  1830                                      <NA>
          Oceania  1875                                      <NA>
          Oceania  1905                                      <NA>
          Oceania  1920                                      <NA>
          Oceania  1940                                      <NA>
    ~ Column mf_adults__25_64_years__percentage_of_tertiary_education (new data, changed data)
+       + New values: 33 / 12248 (0.27%)
          country  year  mf_adults__25_64_years__percentage_of_tertiary_education
          Oceania  1830                                                      <NA>
          Oceania  1875                                                  0.015687
          Oceania  1905                                                  0.382741
          Oceania  1920                                                  2.769854
          Oceania  1940                                                  7.167146
        ~ Changed values: 35 / 12248 (0.29%)
          country  year  mf_adults__25_64_years__percentage_of_tertiary_education -  mf_adults__25_64_years__percentage_of_tertiary_education +
            World  1920                                                     0.77337                                                    0.781506
            World  1940                                                    1.551331                                                    1.576569
            World  1950                                                    2.047779                                                    2.085101
            World  1980                                                    6.050941                                                    6.138223
            World  1985                                                    7.414569                                                    7.512444
    ~ Column mf_youth_and_adults__15_64_years__average_years_of_education (new data, changed data)
+       + New values: 33 / 12248 (0.27%)
          country  year  mf_youth_and_adults__15_64_years__average_years_of_education
          Oceania  1830                                                          <NA>
          Oceania  1875                                                      2.014745
          Oceania  1905                                                      3.511473
          Oceania  1920                                                      4.829186
          Oceania  1940                                                      7.060847
        ~ Changed values: 35 / 12248 (0.29%)
          country  year  mf_youth_and_adults__15_64_years__average_years_of_education -  mf_youth_and_adults__15_64_years__average_years_of_education +
            World  1920                                                        1.637321                                                        1.650328
            World  1940                                                        2.603918                                                        2.623948
            World  1950                                                        3.033632                                                        3.060064
            World  1980                                                        5.310848                                                        5.341849
            World  1985                                                        5.743337                                                        5.771884
    ~ Column mf_youth_and_adults__15_64_years__percentage_of_no_education (new data, changed data)
+       + New values: 33 / 12248 (0.27%)
          country  year  mf_youth_and_adults__15_64_years__percentage_of_no_education
          Oceania  1830                                                          <NA>
          Oceania  1875                                                     46.221493
          Oceania  1905                                                     22.077822
          Oceania  1920                                                     11.920393
          Oceania  1940                                                      4.758517
        ~ Changed values: 35 / 12248 (0.29%)
          country  year  mf_youth_and_adults__15_64_years__percentage_of_no_education -  mf_youth_and_adults__15_64_years__percentage_of_no_education +
            World  1920                                                       69.551849                                                       69.316986
            World  1940                                                       55.364956                                                        55.13752
            World  1950                                                       50.144726                                                       49.903214
            World  1980                                                       31.097197                                                       30.941936
            World  1985                                                       28.678278                                                       28.538013
    ~ Column some_formal_education_both_sexes (new data, changed data)
+       + New values: 33 / 12248 (0.27%)
          country  year  some_formal_education_both_sexes
          Oceania  1830                              <NA>
          Oceania  1875                         53.778507
          Oceania  1905                          77.92218
          Oceania  1920                         88.079605
          Oceania  1940                         95.241486
        ~ Changed values: 35 / 12248 (0.29%)
          country  year  some_formal_education_both_sexes -  some_formal_education_both_sexes +
            World  1920                           30.448151                           30.683014
            World  1940                           44.635044                            44.86248
            World  1950                           49.855274                           50.096786
            World  1980                           68.902802                            69.05806
            World  1985                           71.321724                            71.46199
    ~ Column some_formal_education_female (new data, changed data)
+       + New values: 33 / 12248 (0.27%)
          country  year  some_formal_education_female
          Oceania  1830                          <NA>
          Oceania  1875                     40.652218
          Oceania  1905                     71.107147
          Oceania  1920                     85.529724
          Oceania  1940                     94.274567
        ~ Changed values: 35 / 12248 (0.29%)
          country  year  some_formal_education_female -  some_formal_education_female +
            World  1920                       26.144218                        26.38623
            World  1940                       39.227276                       39.474667
            World  1950                       43.584755                       43.856495
            World  1980                       63.402851                       63.585453
            World  1985                       66.398628                       66.563072
    ~ Column some_formal_education_male (new data)
+       + New values: 33 / 12248 (0.27%)
          country  year  some_formal_education_male
          Oceania  1830                        <NA>
          Oceania  1875                        <NA>
          Oceania  1905                        <NA>
          Oceania  1920                        <NA>
          Oceania  1940                        <NA>
= Dataset garden/education/2023-07-17/education_lee_lee
  = Table education_lee_lee
    ~ Dim country
+       + New values: 33 / 13438 (0.25%)
           year country
           1830 Oceania
           1875 Oceania
           1905 Oceania
           1920 Oceania
           1940 Oceania
    ~ Dim year
+       + New values: 33 / 13438 (0.25%)
          country  year
          Oceania  1830
          Oceania  1875
          Oceania  1905
          Oceania  1920
          Oceania  1940
    ~ Column f_adults__25_64_years__percentage_of_no_education (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  f_adults__25_64_years__percentage_of_no_education
          Oceania  1830                                               <NA>
          Oceania  1875                                          63.042763
          Oceania  1905                                          33.708057
          Oceania  1920                                          17.011372
          Oceania  1940                                           6.374139
        ~ Changed values: 35 / 13438 (0.26%)
          country  year  f_adults__25_64_years__percentage_of_no_education -  f_adults__25_64_years__percentage_of_no_education +
            World  1920                                            76.639046                                            76.396049
            World  1940                                            67.551224                                            67.276291
            World  1950                                              63.2355                                            62.930649
            World  1980                                            42.577347                                            42.365795
            World  1985                                            39.264984                                            39.072979
    ~ Column f_primary_enrollment_rates_combined_wb (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  f_primary_enrollment_rates_combined_wb
          Oceania  1830                                24.55287
          Oceania  1875                               53.097324
          Oceania  1905                                63.63715
          Oceania  1920                               76.452797
          Oceania  1940                               85.292847
        ~ Changed values: 33 / 13438 (0.25%)
          country  year  f_primary_enrollment_rates_combined_wb -  f_primary_enrollment_rates_combined_wb +
            World  1830                                  3.942561                                  3.970571
            World  1875                                 15.653235                                 15.750608
            World  1905                                 25.015865                                 25.158895
            World  1920                                 30.470058                                  30.65745
            World  1940                                 39.993656                                 40.197239
    ~ Column f_secondary_enrollment_rates_combined_wb (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  f_secondary_enrollment_rates_combined_wb
          Oceania  1830                                       0.0
          Oceania  1875                                  0.206406
          Oceania  1905                                  2.299872
          Oceania  1920                                  6.832693
          Oceania  1940                                 17.463144
        ~ Changed values: 33 / 13438 (0.25%)
          country  year  f_secondary_enrollment_rates_combined_wb -  f_secondary_enrollment_rates_combined_wb +
            World  1830                                    0.058195                                    0.058116
            World  1875                                    0.482988                                    0.482269
            World  1905                                    1.274901                                    1.278697
            World  1920                                    2.654305                                    2.671333
            World  1940                                    6.728284                                    6.776528
    ~ Column f_tertiary_enrollment_rates_combined_wb (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  f_tertiary_enrollment_rates_combined_wb
          Oceania  1830                                      0.0
          Oceania  1875                                 0.009471
          Oceania  1905                                 0.148835
          Oceania  1920                                 0.486334
          Oceania  1940                                 1.608257
        ~ Changed values: 33 / 13438 (0.25%)
          country  year  f_tertiary_enrollment_rates_combined_wb -  f_tertiary_enrollment_rates_combined_wb +
            World  1830                                   0.000566                                   0.000565
            World  1875                                   0.008714                                   0.008716
            World  1905                                   0.051624                                   0.051984
            World  1920                                   0.240886                                   0.241886
            World  1940                                   0.726146                                   0.730111
    ~ Column f_youth_and_adults__15_64_years__average_years_of_education (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  f_youth_and_adults__15_64_years__average_years_of_education
          Oceania  1830                                                         <NA>
          Oceania  1875                                                     1.783469
          Oceania  1905                                                     3.448642
          Oceania  1920                                                     4.643779
          Oceania  1940                                                     6.913009
        ~ Changed values: 35 / 13438 (0.26%)
          country  year  f_youth_and_adults__15_64_years__average_years_of_education -  f_youth_and_adults__15_64_years__average_years_of_education +
            World  1920                                                       1.440069                                                       1.453125
            World  1940                                                       2.287405                                                       2.308193
            World  1950                                                       2.629588                                                       2.657026
            World  1980                                                       4.685742                                                       4.719159
            World  1985                                                       5.157459                                                       5.188292
    ~ Column f_youth_and_adults__15_64_years__percentage_of_no_education (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  f_youth_and_adults__15_64_years__percentage_of_no_education
          Oceania  1830                                                         <NA>
          Oceania  1875                                                    59.347782
          Oceania  1905                                                    28.892849
          Oceania  1920                                                    14.470279
          Oceania  1940                                                     5.725431
        ~ Changed values: 35 / 13438 (0.26%)
          country  year  f_youth_and_adults__15_64_years__percentage_of_no_education -  f_youth_and_adults__15_64_years__percentage_of_no_education +
            World  1920                                                      73.855782                                                       73.61377
            World  1940                                                      60.772724                                                      60.525333
            World  1950                                                      56.415245                                                      56.143505
            World  1980                                                      36.597149                                                      36.414547
            World  1985                                                      33.601372                                                      33.436928
    ~ Column female_over_male_enrollment_rates_primary (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  female_over_male_enrollment_rates_primary
          Oceania  1830                                   0.589184
          Oceania  1875                                   0.572207
          Oceania  1905                                   0.651831
          Oceania  1920                                   0.776218
          Oceania  1940                                   0.865063
        ~ Changed values: 33 / 13438 (0.25%)
          country  year  female_over_male_enrollment_rates_primary -  female_over_male_enrollment_rates_primary +
            World  1830                                     0.615492                                     0.615261
            World  1875                                     0.717535                                     0.715941
            World  1905                                     0.768075                                     0.766794
            World  1920                                     0.779362                                      0.77933
            World  1940                                     0.759122                                      0.76001
    ~ Column female_over_male_enrollment_rates_secondary (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  female_over_male_enrollment_rates_secondary
          Oceania  1830                                          0.0
          Oceania  1875                                     0.359109
          Oceania  1905                                     0.717367
          Oceania  1920                                      0.80658
          Oceania  1940                                     0.870306
        ~ Changed values: 33 / 13438 (0.25%)
          country  year  female_over_male_enrollment_rates_secondary -  female_over_male_enrollment_rates_secondary +
            World  1830                                       0.255591                                       0.255576
            World  1875                                       0.328678                                       0.328709
            World  1905                                       0.461879                                       0.462977
            World  1920                                       0.500281                                       0.502269
            World  1940                                       0.788737                                       0.789594
    ~ Column female_over_male_enrollment_rates_tertiary (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  female_over_male_enrollment_rates_tertiary
          Oceania  1830                                         0.0
          Oceania  1875                                    0.051258
          Oceania  1905                                    0.114459
          Oceania  1920                                    0.146292
          Oceania  1940                                    0.424082
        ~ Changed values: 33 / 13438 (0.25%)
          country  year  female_over_male_enrollment_rates_tertiary -  female_over_male_enrollment_rates_tertiary +
            World  1830                                      0.010221                                      0.010218
            World  1875                                      0.040167                                      0.040191
            World  1905                                      0.101375                                      0.101498
            World  1920                                      0.282989                                      0.280839
            World  1940                                      0.562922                                      0.561104
    ~ Column m_primary_enrollment_rates_combined_wb (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  m_primary_enrollment_rates_combined_wb
          Oceania  1830                               41.672661
          Oceania  1875                               92.793945
          Oceania  1905                               97.628227
          Oceania  1920                               98.493958
          Oceania  1940                               98.597321
        ~ Changed values: 34 / 13438 (0.25%)
          country  year  m_primary_enrollment_rates_combined_wb -  m_primary_enrollment_rates_combined_wb +
            World  1825                                  5.672947                                  5.718696
            World  1870                                 18.947872                                 19.128305
            World  1900                                 30.769783                                 31.013018
            World  1915                                 37.475044                                 37.712395
            World  1940                                 52.684086                                 52.890427
    ~ Column m_secondary_enrollment_rates_combined_wb (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  m_secondary_enrollment_rates_combined_wb
          Oceania  1830                                      0.01
          Oceania  1875                                  0.574774
          Oceania  1905                                  3.205992
          Oceania  1920                                  8.471188
          Oceania  1940                                 20.065529
        ~ Changed values: 33 / 13438 (0.25%)
          country  year  m_secondary_enrollment_rates_combined_wb -  m_secondary_enrollment_rates_combined_wb +
            World  1830                                    0.227686                                     0.22739
            World  1875                                    1.469488                                    1.467161
            World  1905                                    2.760252                                    2.761903
            World  1920                                     5.30563                                    5.318531
            World  1940                                    8.530452                                    8.582293
    ~ Column m_tertiary_enrollment_rates_combined_wb (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  m_tertiary_enrollment_rates_combined_wb
          Oceania  1830                                     0.01
          Oceania  1875                                 0.184775
          Oceania  1905                                 1.300337
          Oceania  1920                                 3.324406
          Oceania  1940                                 3.792321
        ~ Changed values: 33 / 13438 (0.25%)
          country  year  m_tertiary_enrollment_rates_combined_wb -  m_tertiary_enrollment_rates_combined_wb +
            World  1830                                    0.05535                                   0.055289
            World  1875                                   0.216945                                   0.216861
            World  1905                                    0.50924                                    0.51217
            World  1920                                   0.851219                                   0.861298
            World  1940                                   1.289959                                   1.301205
    ~ Column mf_adults__25_64_years__percentage_of_tertiary_education (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  mf_adults__25_64_years__percentage_of_tertiary_education
          Oceania  1830                                                      <NA>
          Oceania  1875                                                  0.015687
          Oceania  1905                                                  0.382741
          Oceania  1920                                                  2.769854
          Oceania  1940                                                  7.167146
        ~ Changed values: 35 / 13438 (0.26%)
          country  year  mf_adults__25_64_years__percentage_of_tertiary_education -  mf_adults__25_64_years__percentage_of_tertiary_education +
            World  1920                                                     0.77337                                                    0.781506
            World  1940                                                    1.551331                                                    1.576569
            World  1950                                                    2.047779                                                    2.085101
            World  1980                                                    6.050941                                                    6.138223
            World  1985                                                    7.414569                                                    7.512444
    ~ Column mf_primary_enrollment_rates_combined_wb (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  mf_primary_enrollment_rates_combined_wb
          Oceania  1830                                34.282196
          Oceania  1875                                 82.54821
          Oceania  1905                                97.388832
          Oceania  1920                                98.166855
          Oceania  1940                                96.012329
        ~ Changed values: 33 / 13438 (0.25%)
          country  year  mf_primary_enrollment_rates_combined_wb -  mf_primary_enrollment_rates_combined_wb +
            World  1830                                   5.215985                                   5.255488
            World  1875                                  18.916994                                  19.082468
            World  1905                                  29.000483                                  29.253752
            World  1920                                  35.141056                                    35.3979
            World  1940                                  47.339867                                  47.558609
    ~ Column mf_secondary_enrollment_rates_combined_wb (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  mf_secondary_enrollment_rates_combined_wb
          Oceania  1830                                        0.0
          Oceania  1875                                   0.401571
          Oceania  1905                                    2.76364
          Oceania  1920                                   7.680693
          Oceania  1940                                  18.768568
        ~ Changed values: 33 / 13438 (0.25%)
          country  year  mf_secondary_enrollment_rates_combined_wb -  mf_secondary_enrollment_rates_combined_wb +
            World  1830                                     0.138897                                     0.138708
            World  1875                                     0.975392                                       0.9739
            World  1905                                     2.028203                                     2.030926
            World  1920                                     4.008198                                     4.023164
            World  1940                                     7.626482                                     7.676557
    ~ Column mf_tertiary_enrollment_rates_combined_wb (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  mf_tertiary_enrollment_rates_combined_wb
          Oceania  1830                                       0.0
          Oceania  1875                                  0.105516
          Oceania  1905                                  0.736582
          Oceania  1920                                  1.894581
          Oceania  1940                                  2.712214
        ~ Changed values: 33 / 13438 (0.25%)
          country  year  mf_tertiary_enrollment_rates_combined_wb -  mf_tertiary_enrollment_rates_combined_wb +
            World  1830                                    0.023537                                    0.023505
            World  1875                                    0.109909                                    0.109897
            World  1905                                    0.277638                                    0.279338
            World  1920                                    0.539478                                       0.545
            World  1940                                    1.007502                                    1.015163
    ~ Column mf_youth_and_adults__15_64_years__average_years_of_education (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  mf_youth_and_adults__15_64_years__average_years_of_education
          Oceania  1830                                                          <NA>
          Oceania  1875                                                      2.014745
          Oceania  1905                                                      3.511473
          Oceania  1920                                                      4.829186
          Oceania  1940                                                      7.060847
        ~ Changed values: 35 / 13438 (0.26%)
          country  year  mf_youth_and_adults__15_64_years__average_years_of_education -  mf_youth_and_adults__15_64_years__average_years_of_education +
            World  1920                                                        1.637321                                                        1.650328
            World  1940                                                        2.603918                                                        2.623948
            World  1950                                                        3.033632                                                        3.060064
            World  1980                                                        5.310848                                                        5.341849
            World  1985                                                        5.743337                                                        5.771884
    ~ Column mf_youth_and_adults__15_64_years__percentage_of_no_education (new data, changed data)
+       + New values: 33 / 13438 (0.25%)
          country  year  mf_youth_and_adults__15_64_years__percentage_of_no_education
          Oceania  1830                                                          <NA>
          Oceania  1875                                                     46.221493
          Oceania  1905                                                     22.077822
          Oceania  1920                                                     11.920393
          Oceania  1940                                                      4.758517
        ~ Changed values: 35 / 13438 (0.26%)
          country  year  mf_youth_and_adults__15_64_years__percentage_of_no_education -  mf_youth_and_adults__15_64_years__percentage_of_no_education +
            World  1920                                                       69.551849                                                       69.316986
            World  1940                                                       55.364956                                                        55.13752
            World  1950                                                       50.144726                                                       49.903214
            World  1980                                                       31.097197                                                       30.941936
            World  1985                                                       28.678278                                                       28.538013
~ Dataset garden/faostat/2025-03-17/additional_variables
-   - update_period_days: 365
  = Table cereal_allocation
    ~ Column cereals_allocated_to_animal_feed (changed metadata)
+       +       Food Balance Sheet presents a comprehensive picture of the pattern of a country's food supply during a specified reference period. The food balance sheet shows for each food item - i.e. each primary commodity and a number of processed commodities potentially available for human consumption - the sources of supply and its utilization. The total quantity of foodstuffs produced in a country added to the total quantity imported and adjusted to any change in stocks that may have occurred since the beginning of the reference period gives the supply available during that period. On the utilization side a distinction is made between the quantities exported, fed to livestock, used for seed, put to manufacture for food use and non-food uses, losses during storage and transportation, and food supplies available for human consumption. The per caput supply of each such food item available for human consumption is then obtained by dividing the respective quantity by the related data on the population actually partaking of it. Data on per caput food supplies are expressed in terms of quantity and - by applying appropriate food composition factors for all primary and processed products - also in terms of caloric value and protein and fat content.
-       -       Food Balance Sheet presents a comprehensive picture of the pattern of a country's food supply during a specified reference period.
-       - 
-       -       The food balance sheet shows for each food item - i.e. each primary commodity and a number of processed commodities potentially available for human consumption - the sources of supply and its utilization. The total quantity of foodstuffs produced in a country added to the total quantity imported and adjusted to any change in stocks that may have occurred since the beginning of the reference period gives the supply available during that period. On the utilization side a distinction is made between the quantities exported, fed to livestock, used for seed, put to manufacture for food use and non-food uses, losses during storage and transportation, and food supplies available for human consumption.
-       - 
-       -       The per caput supply of each such food item available for human consumption is then obtained by dividing the respective quantity by the related data on the population actually partaking of it. Data on per caput food supplies are expressed in terms of quantity and - by applying appropriate food composition factors for all primary and processed products - also in terms of caloric value and protein and fat content.
-       -     attribution: Food and Agriculture Organization of the United Nations (2024)
+       +       Food Balance Sheet presents a comprehensive picture of the pattern of a country's food supply during a specified reference period. The food balance sheet shows for each food item - i.e. each primary commodity and a number of processed commodities potentially available for human consumption - the sources of supply and its utilization. The total quantity of foodstuffs produced in a country added to the total quantity imported and adjusted to any change in stocks that may have occurred since the beginning of the reference period gives the supply available during that period. On the utilization side a distinction is made between the quantities exported, fed to livestock, used for seed, put to manufacture for food use and non-food uses, losses during storage and transportation, and food supplies available for human consumption. The per caput supply of each such food item available for human consumption is then obtained by dividing the respective quantity by the related data on the population actually partaking of it. Data on per caput food supplies are expressed in terms of quantity and - by applying appropriate food composition factors for all primary and processed products - also in terms of caloric value and protein and fat content.
-       -       Food Balance Sheet presents a comprehensive picture of the pattern of a country's food supply during a specified reference period.
-       - 
-       -       The food balance sheet shows for each food item - i.e. each primary commodity and a number of processed commodities potentially available for human consumption - the sources of supply and its utilization. The total quantity of foodstuffs produced in a country added to the total quantity imported and adjusted to any change in stocks that may have occurred since the beginning of the reference period gives the supply available during that period. On the utilization side a distinction is made between the quantities exported, fed to livestock, used for seed, put to manufacture for food use and non-food uses, losses during storage and transportation, and food supplies available for human consumption.
-       - 
-       -       The per caput supply of each such food item available for human consumption is then obtained by dividing the respective quantity by the related data on the population actually partaking of it. Data on per caput food supplies are expressed in terms of quantity and - by applying appropriate food composition factors for all primary and processed products - also in terms of caloric value and protein and fat content.
-       -     attribution: Food and Agriculture Organization of the United Nations (2024)
    ~ Column cereals_allocated_to_food (changed metadata)
+       +       Food Balance Sheet presents a comprehensive picture of the pattern of a country's food supply during a specified reference period. The food balance sheet shows for each food item - i.e. each primary commodity and a number of processed commodities potentially available for human consumption - the sources of supply and its utilization. The total quantity of foodstuffs produced in a country added to the total quantity imported and adjusted to any change in stocks that may have occurred since the beginning of the reference period gives the supply available during that period. On the utilization side a distinction is made between the quantities exported, fed to livestock, used for seed, put to manufacture for food use and non-food uses, losses during storage and transportation, and food supplies available for human consumption. The per caput supply of each such food item available for human consumption is then obtained by dividing the respective quantity by the related data on the population actually partaking of it. Data on per caput food supplies are expressed in terms of quantity and - by applying appropriate food composition factors for all primary and processed products - also in terms of caloric value and protein and fat content.
-       -       Food Balance Sheet presents a comprehensive picture of the pattern of a country's food supply during a specified reference period.
-       - 
-       -       The food balance sheet shows for each food item - i.e. each primary commodity and a number of processed commodities potentially available for human consumption - the sources of supply and its utilization. The total quantity of foodstuffs produced in a country added to the total quantity imported and adjusted to any change in stocks that may have occurred since the beginning of the reference period gives the supply available during that period. On the utilization side a distinction is made between the quantities exported, fed to livestock, used for seed, put to manufacture for food use and non-food uses, losses during storage and transportation, and food supplies available for human consumption.
-       - 
-       -       The per caput supply of each such food item available for human consumption is then obtained by dividing the respective quantity by the related data on the population actually partaking of it. Data on per caput food supplies are expressed in terms of quantity and - by applying appropriate food composition factors for all primary and processed products - also in terms of caloric value and protein and fat content.
-       -     attribution: Food and Agriculture Organization of the United Nations (2024)
+       +       Food Balance Sheet presents a comprehensive picture of the pattern of a country's food supply during a specified reference period. The food balance sheet shows for each food item - i.e. each primary commodity and a number of processed commodities potentially available for human consumption - the sources of supply and its utilization. The total quantity of foodstuffs produced in a country added to the total quantity imported and adjusted to any change in stocks that may have occurred since the beginning of the reference period gives the supply available during that period. On the utilization side a distinction is made between the quantities exported, fed to livestock, used for seed, put to manufacture for food use and non-food uses, losses during storage and transportation, and food supplies available for human consumption. The per caput supply of each such food item available for human consumption is then obtained by dividing the respective quantity by the related data on the population actually partaking of it. Data on per caput food supplies are expressed in terms of quantity and - by applying appropriate food composition factors for all primary and processed products - also in terms of caloric value and protein and fat content.
-       -       Food Balance Sheet presents a comprehensive picture of the pattern of a country's food supply during a specified reference period.
-       - 
-       -       The food balance sheet shows for each food item - i.e. each primary commodity and a number of processed commodities potentially available for human consumption - the sources of supply and its utilization. The total quantity of foodstuffs produced in a country added to the total quantity imported and adjusted to any change in stocks that may have occurred since the beginning of the reference period gives the supply available during that period. On the utilization side a distinction is made between the quantities exported, fed to livestock, used for seed, put to manufacture for food use and non-food uses, losses during storage and transportation, and food supplies available for human consumption.
-       - 
-       -       The per caput supply of each such food item available for human consumption is then obtained by dividing the respective quantity by the related data on the population actually partaking of it. Data on per caput food supplies are expressed in terms of quantity and - by applying appropriate food composition factors for all primary and processed products - also in terms of caloric value and protein and fat content.
-       -     attribution: Food and Agriculture Organization of the United Nations (2024)
    ~ Column cereals_allocated_to_other_uses (changed metadata)
+       +       Food Balance Sheet presents a comprehensive picture of the pattern of a country's food supply during a specified reference period. The food balance sheet shows for each food item - i.e. each primary commodity and a number of processed commodities potentially available for human consumption - the sources of supply and its utilization. The total quantity of foodstuffs produced in a country added to the total quantity imported and adjusted to any change in stocks that may have occurred since the beginning of the reference period gives the supply available during that period. On the utilization side a distinction is made between the quantities exported, fed to livestock, used for seed, put to manufacture for food use and non-food uses, losses during storage and transportation, and food supplies available for human consumption. The per caput supply of each such food item available for human consumption is then obtained by dividing the respective quantity by the related data on the population actually partaking of it. Data on per caput food supplies are expressed in terms of quantity and - by applying appropriate food composition factors for all primary and processed products - also in terms of caloric value and protein and fat content.
-       -       Food Balance Sheet presents a comprehensive picture of the pattern of a country's food supply during a specified reference period.
-       - 
-       -       The food balance sheet shows for each food item - i.e. each primary commodity and a number of processed commodities potentially available for human consumption - the sources of supply and its utilization. The total quantity of foodstuffs produced in a country added to the total quantity imported and adjusted to any change in stocks that may have occurred since the beginning of the reference period gives the supply available during that period. On the utilization side a distinction is made between the quantities exported, fed to livestock, used for seed, put to manufacture for food use and non-food uses, losses during storage and transportation, and food supplies available for human consumption.
-       - 
-       -       The per caput supply of each such food item available for human consumption is then obtained by dividing the respective quantity by the related data on the population actually partaking of it. Data on per caput food supplies are expressed in terms of quantity and - by applying appropriate food composition factors for all primary and processed products - also in terms of caloric value and protein and fat content.
-       -     attribution: Food and Agriculture Organization of the United Nations (2024)
+       +       Food Balance Sheet presents a comprehensive picture of the pattern of a country's food supply during a specified reference period. The food balance sheet shows for each food item - i.e. each primary commodity and a number of processed commodities potentially available for human consumption - the sources of supply and its utilization. The total quantity of foodstuffs produced in a country added to the total quantity imported and adjusted to any change in stocks that may have occurred since the beginning of the reference period gives the supply available during that period. On the utilization side a distinction is made between the quantities exported, fed to livestock, used for seed, put to manufacture for food use and non-food uses, losses during storage and transportation, and food supplies available for human consumption. The per caput supply of each such food item available for human consumption is then obtained by dividing the respective quantity by the related data on the population actually partaking of it. Data on per caput food supplies are expressed in terms of quantity and - by applying appropriate food composition factors for all primary and processed products - also in terms of caloric value and protein and fat content.
-       -       Food Balance Sheet presents a comprehensive picture of the pattern of a country's food supply during a specified reference period.
-       - 
-       -       The food balance sheet shows for each food item - i.e. each primary commodity and a number of processed commodities potentially available for human consumption - the sources of supply and its utilization. The total quantity of foodstuffs produced in a country added to the total quantity imported and adjusted to any change in stocks that may have occurred since the beginning of the reference period gives the supply available during that period. On the utilization side a distinction is made between the quantities exported, fed to livestock, used for seed, put to manufacture for food use and non-food uses, losses during storage and transportation, and food supplies available for human consumption.
-       - 
-       -       The per caput supply of each such food item available for human consumption is then obtained by dividing the respective quantity by the related data on the population actually partaking of it. Data on per caput food supplies are expressed in terms of quantity and - by applying appropriate food composition factors for all primary and processed products - also in terms of caloric value and protein and fat content.
-       -     attribution: Food and Agriculture Organization of the United Nations (2024)
    ~ Column share_of_cereals_allocated_to_animal_feed (changed metadata)
+       +       Food Balance Sheet presents a comprehensive picture of the pattern of a country's food supply during a specified reference period. The food balance sheet shows for each food item - i.e. each primary commodity and a number of processed commodities potentially available for human consumption - the sources of supply and its utilization. The total quantity of foodstuffs produced in a country added to the total quantity imported and adjusted to any change in stocks that may have occurred since the beginning of the reference period gives the supply available during that period. On the utilization side a distinction is made between the quantities exported, fed to livestock, used for seed, put to manufacture for food use and non-food uses, losses during storage and transportation, and food supplies available for human consumption. The per caput supply of each such food item available for human consumption is then obtained by dividing the respective quantity by the related data on the population actually partaking of it. Data on per caput food supplies are expressed in terms of quantity and - by applying appropriate food composition factors for all primary and processed products - also in terms of caloric value and protein and fat content.
-       -       Food Balance Sheet presents a comprehensive picture of the pattern of a country's food supply during a specified reference period.
-       - 
-       -       The food balance sheet shows for each food item - i.e. each primary commodity and a number of processed commodities potentially available for human consumption - the sources of supply and its utilization. The total quantity of foodstuffs produced in a country added to the total quantity imported and adjusted to any change in stocks that may have occurred since the beginning of the reference period gives the supply available during that period. On the utilization side a distinction is made between the quantities exported, fed to livestock, used for seed, put to manufacture for food use and non-food uses, losses during storage and transportation, and food supplies available for human consumption.
-       - 
-       -       The per caput supply of each such food item available for human consumption is then obtained by dividing the respective quantity by the related data on the population actually partaking of it. Data on per caput food supplies are expressed in terms of quantity and - by applying appropriate food composition factors for all primary and processed products - also in terms of caloric value and protein and fat content.
-       -     attribution: Food and Agriculture Organization of the United Nations (2024)
+       +       Food Balance Sheet presents a comprehen

...diff too long, truncated...

Automatically updated datasets matching weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk are not included

Edited: 2025-03-25 17:08:25 UTC
Execution time: 1457.42 seconds

@lucasrodes lucasrodes mentioned this pull request Mar 18, 2025
4 tasks
Marigold and others added 14 commits March 18, 2025 16:45
* wip

* add docstring

* sort choices w function, number of indicators per view

* wip

* wip

* add property to get slugs of dimensions in collection

* remove unused import

* 🤖 automatic excess mortality update

* 🤖 update: covid-19 cases and deaths

* 🤖 automatic flunet update

* 🤖 automatic measles update

* 🤖 automatic wildfires update

* fasttrack: fasttrack/latest/test.csv

* 📊 yamagata world (#4149)

* add non_redistributable: true

* 📊 yamagata world

* add in missing countries

* simplify meadow a little

* adding yamagata strain

* remove unused modules

* to grapher date

* use garden flunet

* update var short name

* add zero day date

* 🐝 yamagata dataset title (#4156)

* add non_redistributable: true

* 🐝 yamagata dataset title

* update dataset name

* hack metadata

* wip

* linebreak

* wip

---------

Co-authored-by: owidbot <[email protected]>
Co-authored-by: Fiona Spooner <[email protected]>
Tuna Acisu and others added 7 commits March 24, 2025 11:02
* bring table remixing from grapher into garden

* simplify step

* minor re-structuring, fix minor metadata bug

* align mdim with latest changes

* align explorer with latest changes

* Revert previous merge

---------

Co-authored-by: Tuna Acisu <[email protected]>
@antea04 antea04 marked this pull request as ready for review March 25, 2025 11:25
@antea04 antea04 requested a review from spoonerf March 25, 2025 11:27
Copy link
Contributor

@spoonerf spoonerf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really great work! Just a couple of v minor suggestions. It's a great use of the new tools 👏🏼 👏🏼 👏🏼

@antea04 antea04 merged commit 2547a6b into master Mar 26, 2025
8 checks passed
@antea04 antea04 deleted the data-migration-flow-mdims branch March 26, 2025 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants