Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
commands:builtin:mapfile [2011/10/29 00:43]
ormaaj heading structure
commands:builtin:mapfile [2019/12/05 16:59] (current)
willdye Fixed a minor syntax error (excess right-paren)
Line 1: Line 1:
 ====== The mapfile builtin command ====== ====== The mapfile builtin command ======
-:V4: 
  
 ===== Synopsis ===== ===== Synopsis =====
Line 44: Line 43:
 A very simple example might be to use it as a kind of progress bar. This will print a dot for each line read. Note the escaped comment to hide the appended words from printf. A very simple example might be to use it as a kind of progress bar. This will print a dot for each line read. Note the escaped comment to hide the appended words from printf.
  
-<​code>​$ printf '​%s\n'​ {1..5} | mapfile -c 1 -C '​printf . \#' ​)+<​code>​$ printf '​%s\n'​ {1..5} | mapfile -c 1 -C '​printf . \#'
 .....</​code>​ .....</​code>​
  
Line 73: Line 72:
 Since redirects are syntactically allowed anywhere in a command, we put it before the printf to stay out of the way of additional arguments. Rather than opening "​outfile<​n>"​ for appending on each call by calculating the filename, open an FD for each first and calculate which FD to send output to by measuring the size of x mod 2. The zero-width format specification is used to absorb the index number argument. Since redirects are syntactically allowed anywhere in a command, we put it before the printf to stay out of the way of additional arguments. Rather than opening "​outfile<​n>"​ for appending on each call by calculating the filename, open an FD for each first and calculate which FD to send output to by measuring the size of x mod 2. The zero-width format specification is used to absorb the index number argument.
  
-Another variation might be to add each of these lines to the elements of separate arrays. I'll leave dissecting this one as an exercise for the reader. ​Exploiting array assignment like this is somewhat obscure ​and potentially dangerous behavior.+Another variation might be to add each of these lines to the elements of separate arrays. I'll leave dissecting this one as an exercise for the reader. ​This is quite the hack but illustrates some interesting properties of printf -v and mapfile -C (which you should probably never use in real code).
  
-<​code>​$ y=( 'a[j]' 'b[j++]' ); printf '​input%s\n'​ {1..10} | mapfile -tc 1 -C '​printf -v "${y[${#a[@]} + ${#b[@]} ) % 2]}" -- "​%.sprefix %s"'​ x +<​code>​$ y=( 'odd[j]' 'even[j++]' ); printf '​input%s\n'​ {1..10} | mapfile -tc 1 -C '​printf -v "​${y[${#​x[@]} % 2]}" -- "​%.sprefix %s"'​ xprintf '%s\n' "${odd[@]}" ​'' ​"${even[@]}"; }
-printf '%b\n' "${a[@]}\n" "${b[@]}"+
 prefix input1 prefix input1
 prefix input3 prefix input3
Line 90: Line 88:
 </​code>​ </​code>​
  
-===== Bugs =====+This example based on yet another #bash question illustrates mapfile in combination with read. The sample input is the heredoc to ''​main''​. The goal is to build a "​struct"​ based upon records in the input file made up of the numbers following the colon on each line. Every 3rd line is a key followed by 2 corresponding fields. The showRecord function takes a key and returns the record.
  
-  ​Early implementations were buggy. For example, ''​mapfile'' ​filling the readline history buffer with calls to the ''​CALLBACK''​. ​This was fixed in 4.1 betaSome bugs may still exist.+<​code>​ 
 +#​!/​usr/​bin/​env bash 
 + 
 +showRecord() { 
 +    printf '​key[%d] = %d, %d\n' "​$1"​ "​${vals[@]:​keys[$1]*2:​2}"​ 
 +
 + 
 +parseRecords() { 
 +    trap 'unset -f _f' RETURN 
 +    _f() { 
 +        local x 
 +        IFS=: read -r _ x 
 +        ((keys[x]=n++)) 
 +    } 
 +    local n 
 + 
 +    _f 
 +    mapfile -tc2 -C _f "​$1"​ 
 +    eval "​$1"'​=("​${'"​$1"'​[@]##​*:​}"​)'​ # Return the array with some modification 
 +
 + 
 +main() { 
 +    local -a keys vals 
 +    parseRecords vals 
 +    showRecord "​$1"​ 
 +
 + 
 +main "​$1"​ <<​-"​EOF"​ 
 +fabric.domain:​123 
 +routex:1 
 +routey:2 
 +fabric.domain:​321 
 +routex:6 
 +routey:4 
 +EOF 
 +</​code>​ 
 + 
 +For example, ​running ''​scriptname 321''​ would output ''​key[321] = 6, 4''​. Every 2 lines read by ''​mapfile''​the function ​''​_f'' ​is called, which reads one additional lineSince the first line in the file is a key, and ''​_f''​ is responsible for the keys, it gets called first so that ''​mapfile''​ starts by reading the second line of input, calling ''​_f''​ with each subsequent 2 iterationsThe RETURN trap is unimportant. 
 +===== Bugs =====
  
 +  * Early implementations were buggy. For example, ''​mapfile''​ filling the readline history buffer with calls to the ''​CALLBACK''​. This was fixed in 4.1 beta.
 +  * ''​mapfile -n''​ reads an extra line beyond the last line assigned to the array, through Bash. [[ftp://​ftp.gnu.org/​gnu/​bash/​bash-4.2-patches/​bash42-035 | Fixed in 4.2.35]].
 +  * ''​mapfile''​ callbacks could cause a crash if the variable being assigned is manipulated in certain ways. [[https://​lists.gnu.org/​archive/​html/​bug-bash/​2013-01/​msg00039.html]]. Fixed in 4.3.
 ===== To Do ===== ===== To Do =====
   * Create an implementation as a shell function that's portable between Ksh, Zsh, and Bash (and possibly other bourne-like shells with array support).   * Create an implementation as a shell function that's portable between Ksh, Zsh, and Bash (and possibly other bourne-like shells with array support).