How to solve “Broken Pipe” error when using awk with headpiping find to rsyncryptoFile name with spaces...
Is this animal really missing?
Force user to remove USB token
Provisioning profile doesn't include the application-identifier and keychain-access-groups entitlements
Why doesn't the EU now just force the UK to choose between referendum and no-deal?
Counter-example to the existence of left Bousfield localization of combinatorial model category
Replacing Windows 7 security updates with anti-virus?
When were linguistics departments first established
Humans have energy, but not water. What happens?
What wound would be of little consequence to a biped but terrible for a quadruped?
Why would a jet engine that runs at temps excess of 2000°C burn when it crashes?
Can the druid cantrip Thorn Whip really defeat a water weird this easily?
Running a subshell from the middle of the current command
The meaning of the "at the of"
Decoding assembly instructions in a Game Boy disassembler
When is a batch class instantiated when you schedule it?
"One can do his homework in the library"
Deleting missing values from a dataset
It's a yearly task, alright
Can "semicircle" be used to refer to a part-circle that is not a exact half-circle?
Ban on all campaign finance?
What is the blue range indicating on this manifold pressure gauge?
What is the difference between "shut" and "close"?
What Happens when Passenger Refuses to Fly Boeing 737 Max?
Best approach to update all entries in a list that is paginated?
How to solve “Broken Pipe” error when using awk with head
piping find to rsyncryptoFile name with spaces piped to two xargs commandsHow to pipe awk output (with periodic, continuous input) to output file?Error 32 (Broken Pipe)using awk with parallelusing awk with find -execIs there a way around broken pipe?Copy matches of recursive search to a new locationchunking screen output using grep/head/awk/sed/etcloop inside multiple-level paths and rename
I'm getting broken pipe errors from a command that does something like:
ls -tr1 /a/path | awk -F 'n' -vpath=/prepend/path/ '{print path$1}' | head -n 50
Essentially I want to list (with absolute path) the oldest X files in a directory.
What seems to happen is that the output is correct (I get 50 file paths output) but that when head has output the 50 files it closes stdin causing awk to throw a broken pipe error as it is still outputting more rows.
linux pipe awk head
bumped to the homepage by Community♦ 10 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
I'm getting broken pipe errors from a command that does something like:
ls -tr1 /a/path | awk -F 'n' -vpath=/prepend/path/ '{print path$1}' | head -n 50
Essentially I want to list (with absolute path) the oldest X files in a directory.
What seems to happen is that the output is correct (I get 50 file paths output) but that when head has output the 50 files it closes stdin causing awk to throw a broken pipe error as it is still outputting more rows.
linux pipe awk head
bumped to the homepage by Community♦ 10 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
Please answer your question using the answer your question button below as soon as you can! Then you can even get reputation for it and accept it.
– slhck
Nov 28 '11 at 17:11
I would have, but as a new user I can't do that for another... 5 hours apparently. I wanted to write the answer down so I didn't forget.
– Jon
Nov 28 '11 at 17:11
add a comment |
I'm getting broken pipe errors from a command that does something like:
ls -tr1 /a/path | awk -F 'n' -vpath=/prepend/path/ '{print path$1}' | head -n 50
Essentially I want to list (with absolute path) the oldest X files in a directory.
What seems to happen is that the output is correct (I get 50 file paths output) but that when head has output the 50 files it closes stdin causing awk to throw a broken pipe error as it is still outputting more rows.
linux pipe awk head
I'm getting broken pipe errors from a command that does something like:
ls -tr1 /a/path | awk -F 'n' -vpath=/prepend/path/ '{print path$1}' | head -n 50
Essentially I want to list (with absolute path) the oldest X files in a directory.
What seems to happen is that the output is correct (I get 50 file paths output) but that when head has output the 50 files it closes stdin causing awk to throw a broken pipe error as it is still outputting more rows.
linux pipe awk head
linux pipe awk head
edited Nov 28 '11 at 17:12
slhck
162k47448470
162k47448470
asked Nov 28 '11 at 15:23
JonJon
63
63
bumped to the homepage by Community♦ 10 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 10 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
Please answer your question using the answer your question button below as soon as you can! Then you can even get reputation for it and accept it.
– slhck
Nov 28 '11 at 17:11
I would have, but as a new user I can't do that for another... 5 hours apparently. I wanted to write the answer down so I didn't forget.
– Jon
Nov 28 '11 at 17:11
add a comment |
Please answer your question using the answer your question button below as soon as you can! Then you can even get reputation for it and accept it.
– slhck
Nov 28 '11 at 17:11
I would have, but as a new user I can't do that for another... 5 hours apparently. I wanted to write the answer down so I didn't forget.
– Jon
Nov 28 '11 at 17:11
Please answer your question using the answer your question button below as soon as you can! Then you can even get reputation for it and accept it.
– slhck
Nov 28 '11 at 17:11
Please answer your question using the answer your question button below as soon as you can! Then you can even get reputation for it and accept it.
– slhck
Nov 28 '11 at 17:11
I would have, but as a new user I can't do that for another... 5 hours apparently. I wanted to write the answer down so I didn't forget.
– Jon
Nov 28 '11 at 17:11
I would have, but as a new user I can't do that for another... 5 hours apparently. I wanted to write the answer down so I didn't forget.
– Jon
Nov 28 '11 at 17:11
add a comment |
1 Answer
1
active
oldest
votes
Solution from the OP, revision 2
Turns out I was being pretty dumb.
Firstly there is no need to have awk prepend the path to every single file just to throw most of it away. So the awk statement should be the last pipe.
Secondly instead of reversing sorting with ls we can do a standard time sort and use tail to extract the lines we're after. This ensure the pipe remains open for the entire process.
The new command would look like:
ls -t1 /a/path | tail -n 50 | awk -F 'n' -vpath=/prepend/path/ '{print path$1}'
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f362221%2fhow-to-solve-broken-pipe-error-when-using-awk-with-head%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Solution from the OP, revision 2
Turns out I was being pretty dumb.
Firstly there is no need to have awk prepend the path to every single file just to throw most of it away. So the awk statement should be the last pipe.
Secondly instead of reversing sorting with ls we can do a standard time sort and use tail to extract the lines we're after. This ensure the pipe remains open for the entire process.
The new command would look like:
ls -t1 /a/path | tail -n 50 | awk -F 'n' -vpath=/prepend/path/ '{print path$1}'
add a comment |
Solution from the OP, revision 2
Turns out I was being pretty dumb.
Firstly there is no need to have awk prepend the path to every single file just to throw most of it away. So the awk statement should be the last pipe.
Secondly instead of reversing sorting with ls we can do a standard time sort and use tail to extract the lines we're after. This ensure the pipe remains open for the entire process.
The new command would look like:
ls -t1 /a/path | tail -n 50 | awk -F 'n' -vpath=/prepend/path/ '{print path$1}'
add a comment |
Solution from the OP, revision 2
Turns out I was being pretty dumb.
Firstly there is no need to have awk prepend the path to every single file just to throw most of it away. So the awk statement should be the last pipe.
Secondly instead of reversing sorting with ls we can do a standard time sort and use tail to extract the lines we're after. This ensure the pipe remains open for the entire process.
The new command would look like:
ls -t1 /a/path | tail -n 50 | awk -F 'n' -vpath=/prepend/path/ '{print path$1}'
Solution from the OP, revision 2
Turns out I was being pretty dumb.
Firstly there is no need to have awk prepend the path to every single file just to throw most of it away. So the awk statement should be the last pipe.
Secondly instead of reversing sorting with ls we can do a standard time sort and use tail to extract the lines we're after. This ensure the pipe remains open for the entire process.
The new command would look like:
ls -t1 /a/path | tail -n 50 | awk -F 'n' -vpath=/prepend/path/ '{print path$1}'
edited Mar 20 '17 at 10:04
community wiki
2 revs
slhck
add a comment |
add a comment |
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f362221%2fhow-to-solve-broken-pipe-error-when-using-awk-with-head%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Please answer your question using the answer your question button below as soon as you can! Then you can even get reputation for it and accept it.
– slhck
Nov 28 '11 at 17:11
I would have, but as a new user I can't do that for another... 5 hours apparently. I wanted to write the answer down so I didn't forget.
– Jon
Nov 28 '11 at 17:11