How to solve “Broken Pipe” error when using awk with headpiping find to rsyncryptoFile name with spaces...

Is this animal really missing?

Force user to remove USB token

Provisioning profile doesn't include the application-identifier and keychain-access-groups entitlements

Why doesn't the EU now just force the UK to choose between referendum and no-deal?

Counter-example to the existence of left Bousfield localization of combinatorial model category

Replacing Windows 7 security updates with anti-virus?

When were linguistics departments first established

Humans have energy, but not water. What happens?

What wound would be of little consequence to a biped but terrible for a quadruped?

Why would a jet engine that runs at temps excess of 2000°C burn when it crashes?

Can the druid cantrip Thorn Whip really defeat a water weird this easily?

Running a subshell from the middle of the current command

The meaning of the "at the of"

Decoding assembly instructions in a Game Boy disassembler

When is a batch class instantiated when you schedule it?

"One can do his homework in the library"

Deleting missing values from a dataset

It's a yearly task, alright

Can "semicircle" be used to refer to a part-circle that is not a exact half-circle?

Ban on all campaign finance?

What is the blue range indicating on this manifold pressure gauge?

What is the difference between "shut" and "close"?

What Happens when Passenger Refuses to Fly Boeing 737 Max?

Best approach to update all entries in a list that is paginated?



How to solve “Broken Pipe” error when using awk with head


piping find to rsyncryptoFile name with spaces piped to two xargs commandsHow to pipe awk output (with periodic, continuous input) to output file?Error 32 (Broken Pipe)using awk with parallelusing awk with find -execIs there a way around broken pipe?Copy matches of recursive search to a new locationchunking screen output using grep/head/awk/sed/etcloop inside multiple-level paths and rename













1















I'm getting broken pipe errors from a command that does something like:



ls -tr1 /a/path | awk -F 'n' -vpath=/prepend/path/ '{print path$1}' | head -n 50


Essentially I want to list (with absolute path) the oldest X files in a directory.



What seems to happen is that the output is correct (I get 50 file paths output) but that when head has output the 50 files it closes stdin causing awk to throw a broken pipe error as it is still outputting more rows.










share|improve this question
















bumped to the homepage by Community 10 mins ago


This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
















  • Please answer your question using the answer your question button below as soon as you can! Then you can even get reputation for it and accept it.

    – slhck
    Nov 28 '11 at 17:11











  • I would have, but as a new user I can't do that for another... 5 hours apparently. I wanted to write the answer down so I didn't forget.

    – Jon
    Nov 28 '11 at 17:11


















1















I'm getting broken pipe errors from a command that does something like:



ls -tr1 /a/path | awk -F 'n' -vpath=/prepend/path/ '{print path$1}' | head -n 50


Essentially I want to list (with absolute path) the oldest X files in a directory.



What seems to happen is that the output is correct (I get 50 file paths output) but that when head has output the 50 files it closes stdin causing awk to throw a broken pipe error as it is still outputting more rows.










share|improve this question
















bumped to the homepage by Community 10 mins ago


This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
















  • Please answer your question using the answer your question button below as soon as you can! Then you can even get reputation for it and accept it.

    – slhck
    Nov 28 '11 at 17:11











  • I would have, but as a new user I can't do that for another... 5 hours apparently. I wanted to write the answer down so I didn't forget.

    – Jon
    Nov 28 '11 at 17:11
















1












1








1


1






I'm getting broken pipe errors from a command that does something like:



ls -tr1 /a/path | awk -F 'n' -vpath=/prepend/path/ '{print path$1}' | head -n 50


Essentially I want to list (with absolute path) the oldest X files in a directory.



What seems to happen is that the output is correct (I get 50 file paths output) but that when head has output the 50 files it closes stdin causing awk to throw a broken pipe error as it is still outputting more rows.










share|improve this question
















I'm getting broken pipe errors from a command that does something like:



ls -tr1 /a/path | awk -F 'n' -vpath=/prepend/path/ '{print path$1}' | head -n 50


Essentially I want to list (with absolute path) the oldest X files in a directory.



What seems to happen is that the output is correct (I get 50 file paths output) but that when head has output the 50 files it closes stdin causing awk to throw a broken pipe error as it is still outputting more rows.







linux pipe awk head






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 28 '11 at 17:12









slhck

162k47448470




162k47448470










asked Nov 28 '11 at 15:23









JonJon

63




63





bumped to the homepage by Community 10 mins ago


This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.







bumped to the homepage by Community 10 mins ago


This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.















  • Please answer your question using the answer your question button below as soon as you can! Then you can even get reputation for it and accept it.

    – slhck
    Nov 28 '11 at 17:11











  • I would have, but as a new user I can't do that for another... 5 hours apparently. I wanted to write the answer down so I didn't forget.

    – Jon
    Nov 28 '11 at 17:11





















  • Please answer your question using the answer your question button below as soon as you can! Then you can even get reputation for it and accept it.

    – slhck
    Nov 28 '11 at 17:11











  • I would have, but as a new user I can't do that for another... 5 hours apparently. I wanted to write the answer down so I didn't forget.

    – Jon
    Nov 28 '11 at 17:11



















Please answer your question using the answer your question button below as soon as you can! Then you can even get reputation for it and accept it.

– slhck
Nov 28 '11 at 17:11





Please answer your question using the answer your question button below as soon as you can! Then you can even get reputation for it and accept it.

– slhck
Nov 28 '11 at 17:11













I would have, but as a new user I can't do that for another... 5 hours apparently. I wanted to write the answer down so I didn't forget.

– Jon
Nov 28 '11 at 17:11







I would have, but as a new user I can't do that for another... 5 hours apparently. I wanted to write the answer down so I didn't forget.

– Jon
Nov 28 '11 at 17:11












1 Answer
1






active

oldest

votes


















0














Solution from the OP, revision 2





Turns out I was being pretty dumb.



Firstly there is no need to have awk prepend the path to every single file just to throw most of it away. So the awk statement should be the last pipe.



Secondly instead of reversing sorting with ls we can do a standard time sort and use tail to extract the lines we're after. This ensure the pipe remains open for the entire process.



The new command would look like:



ls -t1 /a/path | tail -n 50 | awk -F 'n' -vpath=/prepend/path/ '{print path$1}'





share|improve this answer

























    Your Answer








    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "3"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f362221%2fhow-to-solve-broken-pipe-error-when-using-awk-with-head%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0














    Solution from the OP, revision 2





    Turns out I was being pretty dumb.



    Firstly there is no need to have awk prepend the path to every single file just to throw most of it away. So the awk statement should be the last pipe.



    Secondly instead of reversing sorting with ls we can do a standard time sort and use tail to extract the lines we're after. This ensure the pipe remains open for the entire process.



    The new command would look like:



    ls -t1 /a/path | tail -n 50 | awk -F 'n' -vpath=/prepend/path/ '{print path$1}'





    share|improve this answer






























      0














      Solution from the OP, revision 2





      Turns out I was being pretty dumb.



      Firstly there is no need to have awk prepend the path to every single file just to throw most of it away. So the awk statement should be the last pipe.



      Secondly instead of reversing sorting with ls we can do a standard time sort and use tail to extract the lines we're after. This ensure the pipe remains open for the entire process.



      The new command would look like:



      ls -t1 /a/path | tail -n 50 | awk -F 'n' -vpath=/prepend/path/ '{print path$1}'





      share|improve this answer




























        0












        0








        0







        Solution from the OP, revision 2





        Turns out I was being pretty dumb.



        Firstly there is no need to have awk prepend the path to every single file just to throw most of it away. So the awk statement should be the last pipe.



        Secondly instead of reversing sorting with ls we can do a standard time sort and use tail to extract the lines we're after. This ensure the pipe remains open for the entire process.



        The new command would look like:



        ls -t1 /a/path | tail -n 50 | awk -F 'n' -vpath=/prepend/path/ '{print path$1}'





        share|improve this answer















        Solution from the OP, revision 2





        Turns out I was being pretty dumb.



        Firstly there is no need to have awk prepend the path to every single file just to throw most of it away. So the awk statement should be the last pipe.



        Secondly instead of reversing sorting with ls we can do a standard time sort and use tail to extract the lines we're after. This ensure the pipe remains open for the entire process.



        The new command would look like:



        ls -t1 /a/path | tail -n 50 | awk -F 'n' -vpath=/prepend/path/ '{print path$1}'






        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited Mar 20 '17 at 10:04


























        community wiki





        2 revs
        slhck































            draft saved

            draft discarded




















































            Thanks for contributing an answer to Super User!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f362221%2fhow-to-solve-broken-pipe-error-when-using-awk-with-head%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Why not use the yoke to control yaw, as well as pitch and roll? Announcing the arrival of...

            Couldn't open a raw socket. Error: Permission denied (13) (nmap)Is it possible to run networking commands...

            VNC viewer RFB protocol error: bad desktop size 0x0I Cannot Type the Key 'd' (lowercase) in VNC Viewer...