Split large files in small chunks using "split" command
How to split big size files in small pieces in linux and unix?
I have seen sometimes while working on Linux and/or Unix system / server, we want to convert large files (big in size) into small chunks of files. Even sometimes a large file can take long time to open up. That can also increase load on server. Specially when a file is couple of GB in size and you want to debug something quickly in it, its better to split it in small chunks to speed up your work.
We can use readyily available “split” command in Linux or Unix.
To use this command, we can use following few switches with that command.
Usage: split [OPTION] [INPUT [PREFIX]]
Output fixed-size pieces of INPUT to PREFIXaa, PREFIXab, ...; default
size is 1000 lines, and default PREFIX is `x'. With no INPUT, or when INPUT
is -, read standard input.
Mandatory arguments to long options are mandatory for short options too.
-a, --suffix-length=N use suffixes of length N (default 2)
-b, --bytes=SIZE put SIZE bytes per output file
-C, --line-bytes=SIZE put at most SIZE bytes of lines per output file
-d, --numeric-suffixes use numeric suffixes instead of alphabetic
-l, --lines=NUMBER put NUMBER lines per output file
--verbose print a diagnostic to standard error just
before each output file is opened
--help display this help and exit
--version output version information and exit
SIZE may have a multiplier suffix: b for 512, k for 1K, m for 1 Meg.
Report bugs to
E.g we have one file with ~500MB in size and we want to split it in 100 MB multiple files.
Then we can use following command.
andy@andy-desktop:~$ du -sh squid-log-file
Now this is 489 MB size file and we want to convert it into 100MB size files.
- andy@andy-desktop:~$ split -b 100M squid-log-file
This above command will create 5 files. Those files names will start from xa...
e.g., on my computer this above command had created “xaa xab xac xad xae” files.
andy@andy-desktop:~$ du -sh xa*
Isn't it clever !!!!!
This will help system administrator / desktop users and any body who want this functionlity.
To know more details and description of "split" command you can use man page for split command.
- andy@andy-desktop:~$ man split