Data notes

Some notes written up in the form of Jupyter notebooks. These are fairly rough, often written when I am trying to understand a new idea, or experimenting with a new package or technique, but may be of use to others too. See Hamel Husain’s excellent NBSanity project that renders these notebooks using Quarto.

Data tools

• DuckDB experiments (i.e., DuckDB is really fast)
• Comparison of Polars vs Pandas
• How datetime values work in Pandas

Geospatial

• Creating hexgrids for geospatial data
• Creating shapefiles for uniform geometries (e.g., regular hexgrids, square grids)
• Converting population data between geometries

Machine Learning

• Active inference in machine learning
• Fine-tuning transformer embeddings for entity resolution of clinical data

Statistics

• Statistical performance measures (e.g., PPV, Sensitivity, Specificity, Positive and Negative Likelihood Ratios)

Webscraping

• Intro to webscraping