Tag Archives: design data pipeline

Referring to Columns in R by Name Rather than Number has Pros and Cons

There are different ways to refer to variables in R dataframes. You can use a field names, and you can also use field numbers.

Referring to columns in R can be done using both number and field name syntax. Although field name syntax is easier to use in programming, my blog demonstrates how you can use column numbers to make automation easier.

Two Takeaways from Danny Ma’s Machine Learning Panel: Understanding the Problem, and Understanding your Data

Roller coaster like an ETL pipeline that does automation

This lively panel discussed many topics around designing and implementing machine learning pipelines. Two main issues were identified. The first is that you really have to take some time to do exploratory research and define the problem. The second is that you need to also understand the business rules and context behind the data.