New ask Hacker News story: Ask HN: What “ancient” prog. languages to include in my data pipeline?
Ask HN: What “ancient” prog. languages to include in my data pipeline?
4 by m_kos | 1 comments on Hacker News.
I am writing a data processing and analysis pipeline for my dissertation. I thought it would be entertaining to re-write a few tiniest bits of my Python code in older languages, like COBOL or LISP, so that I can cite them in future publications. (And it would be fun to write a few lines of code in them!) By the "tiniest bits" I mean even just saving a list of all files in a directory to a text file, concatenating multiple files, or renaming them. Questions : 1) Would you have any recommendations on what languages would be easy to set up and use (I am on Windows 10)? 2) What tasks would those languages be best for? The pipeline : - configures data collection protocol by interacting with Firebase Realtime Database (NoSQL) - fetches collected data from Firebase Storage every day (thousands of tiny CSV files) - backs up the data - decrypts the files and transforms the data (multidimensional time-series) across multiple files to prepare for analysis and visualization - computes various metrics based on the data and visualizes them (the end goal is to have a data monitoring dashboard with alerts) - re-configures data collection protocol based on the metrics P.S. If any seasoned engineer would be willing to very occasionally give me - a grad student - some pointers on best software development practices, I'd love to hear back from you. I am primarily self-taught, so I am surely doing many things incorrectly.
4 by m_kos | 1 comments on Hacker News.
I am writing a data processing and analysis pipeline for my dissertation. I thought it would be entertaining to re-write a few tiniest bits of my Python code in older languages, like COBOL or LISP, so that I can cite them in future publications. (And it would be fun to write a few lines of code in them!) By the "tiniest bits" I mean even just saving a list of all files in a directory to a text file, concatenating multiple files, or renaming them. Questions : 1) Would you have any recommendations on what languages would be easy to set up and use (I am on Windows 10)? 2) What tasks would those languages be best for? The pipeline : - configures data collection protocol by interacting with Firebase Realtime Database (NoSQL) - fetches collected data from Firebase Storage every day (thousands of tiny CSV files) - backs up the data - decrypts the files and transforms the data (multidimensional time-series) across multiple files to prepare for analysis and visualization - computes various metrics based on the data and visualizes them (the end goal is to have a data monitoring dashboard with alerts) - re-configures data collection protocol based on the metrics P.S. If any seasoned engineer would be willing to very occasionally give me - a grad student - some pointers on best software development practices, I'd love to hear back from you. I am primarily self-taught, so I am surely doing many things incorrectly.
Comments
Post a Comment