I had a unix directory with multiple CSV files, all with the same prefix. I needed them to be concatenated together to make one large file using a shell script.
I used the below code, copied multiple times (one for each file prefix)
if [ -f $new_file ] ; then
for file in /dir/file?*.csv
tail -n +2 “$file” >> $new_file
Here is the code with comments:
2 if [ -f $new_file ] ; then
3 rm $new_file
5 for file in /dir/file?*.csv
7 tail -n +2 “$file” >> $new_file
1 – Name of the file you are creating
2 – 4 Check if the file already exists. If so, then remove it
5 Loop through the smaller files. Note that I have a ? to make sure that it is not trying to add the big file back to itself.
6-8 Concatenate the file to the $new_file, but ONLY lines 2-end of file. This omits the header record of the .csv file.