The Split-Apply-Combine Technique for Machine Learning with R

Introduction Much discussion in the R community has revolved around the proper way to implement the “split-apply-combine”. In particular, I love the exploration of this topic in this blog post. It seems that the “preferred” approach is dplyr::group_by() + tidyr::nest() for splitting, dplyr::mutate() + purrr::map() for applying, and tidyr::unnest() for combining. Additionally, many in the community have shown implementations of the “many models” approach in {tidyverse}-style pipelines, often also using the {broom} package.

Read more

Share Comments

Fuzzy Matching with Texas High School Academic Competition Results and SAT/ACT Scores

Introduction As a follow-up to [a previous post]((https://tonyelhabr.rbind.io/posts/correlations-texas-high-school-academics/) about correlations between Texas high school academic UIL competition scores and SAT/ACT scores, I wanted explore some of the “alternatives” to joining the two data sets—which come from different sources. In that post, I simply perform a an inner_join() using the school and city names as keys. While this decision ensures that the data integrity is “high”, there are potentially many un-matched schools that could have been included in the analysis with some sound “fuzzy matching”.

Read more

Share Comments

Visualizing Texas High School SAT Math Scores with Bubble Grids

Two awesome things inspired this post: {ggplot2}’s version 3.0 release on CRAN, including full support for the {sf} package and new functions geom_sf() and coord_sf(), which make plotting data from shapefiles very straightforward. Jonas Scholey’s blog post discussing the use of “bubble grid” maps as an alternative to choropleth maps, which seem to be used more prevalent. As Jonas implies, using color as a visual encoding is not always the best option, a notion with which I strongly agree.

Read more

Share Comments

Correlations Between Texas High School Academic Competition Results and SAT/ACT Scores

Introduction I wanted to do a follow-up on my series of posts about Texas high school University Interscholastic League (UIL) academic competitions to more closely evaluate the relationship between the school performance in those competitions with school-wide SAT) and ACT scores. For those who may not be familiar with these tests, these are the two most popular standardized tests used for college admission in the United States. In my introduction to that series, I stated the following: School-wide … scores on state- and national-standardized tests (e.

Read more

Share Comments

An Analysis of Texas High School Academic Competition Results, Part 1 - Introduction

Show-- NOTE: This is part of a series of posts discussing my findings of Texas high school academic University Interscholastic Scholarship (UIL) competitions. To keep this and the other write-ups concise and to focus reader attention on the content, I have decided not to show the underlying code (especially that which is used to create the visuals). Nonetheless, the full code can be viewed on my GitHub account. In the future, I may write some some kind of addendum to demonstrate some of the programming implementation that I think is notable in some way.

Read more

Share Comments

An Analysis of Texas High School Academic Competition Results, Part 2 - Competitions

Competition Participation Some of the first questions that might come to mind are those regarding the number of schools in each level of competition (District, Region, and State) and each conference classification level (1A, 2A, … 6A). It seems fair to say that the distribution of schools among Districts, Regions, and Conferences is relatively even. 1 2 This is to be expected since the UIL presumably tries to divide schools evenly among each grouping (to the extent possible) in order to stimulate fair competition.

Read more

Share Comments

An Analysis of Texas High School Academic Competition Results, Part 3 - Individuals

Let’s take a look at individual competitors in the academic UIL competitions. Individual Participation The first question that comes to mind is that of participation–which individuals have competed the most? NOTE: To give some context to the values for individual participants, I’ll include the numbers for myself (“Elhabr, Anthony”) in applicable contexts. rnk name school city conf n 1 Jansa, Wade GARDEN CITY GARDEN CITY 1 57 2 Chen, Kevin CLEMENTS SUGAR LAND 5 56 3 Hanson, Dillon LINDSAY LINDSAY 1 53 4 Gee, John CALHOUN PORT LAVACA 4 47 5 Zhang, Mark CLEMENTS SUGAR LAND 5 47 6 Robertson, Nick BRIDGE CITY BRIDGE CITY 3 46 7 Ryan, Alex KLEIN KLEIN 5 46 8 Strelke, Nick ARGYLE ARGYLE 3 45 9 Niehues, Taylor GARDEN CITY GARDEN CITY 1 44 10 Bass, Michael SPRING HILL LONGVIEW 3 43 1722 Elhabr, Anthony CLEMENS SCHERTZ 4 13 1 # of total rows: 123,409 Although the names here may not provide much insight, the counts provide some context regarding the limits of individual participation.

Read more

Share Comments

An Analysis of Texas High School Academic Competition Results, Part 4 - Schools

Having investigated individuals elsewhere, let’s now take a look at the schools. NOTE: Although I began the examinations of competitions and individuals by looking at volume of participation (to provide context), I’ll skip an analogous discussion here because the participation of schools is shown indirectly through those analyses.) School Scores Let’s begin by looking at some of the same metrics shown for individual students, but aggregated across all students for each school.

Read more

Share Comments

An Analysis of Texas High School Academic Competition Results, Part 5 - Miscellaneous

There’s a lot to analyze with the Texas high school academic UIL data set. Maybe I find it more interesting than others due to my personal experiences with these competitions. Now, after examining some of the biggest topics associated with this data–including competitions, individuals, and schools–in a broad manner, there are some other things that don’t necessarily fall into these categories that I think are worth investigating. Siblings Let’s look at the performance of siblings.

Read more

Share Comments

The DRY Principle and Knowing When to Make a Package

Don’t Repeat Yourself (DRY) Probably everyone who has done some kind of programming has heard of the Don’t Repeat Yourself (DRY) principle. In a nutshell, it’s about reducing code redundancy for the purpose of reducing error and enhancing readability. Undoubtedly the most common manifestation of the DRY principle is the creation of a function for re-used logic. The “rule of 3” is a good shorthand for identifying when you might want to rethink how your code is organized– “You should consider writing a function whenever you’ve copied and pasted a block of code more than twice (i.

Read more

Share Comments