Downloading GitHub Archive files using wget: has {} for iterationProblems downloading files with...

How can my powered armor quickly replace its ceramic plates?

Why am I able to open Wireshark in macOS without root privileges?

A starship is travelling at 0.9c and collides with a small rock. Will it leave a clean hole through, or will more happen?

What would be the rarity of this magic item(s)?

False written accusations not made public - is there law to cover this?

Early credit roll before the end of the film

Why publish a research paper when a blog post or a lecture slide can have more citation count than a journal paper?

Is it possible to grant users sftp access without shell access? If yes, how is it implemented?

If I delete my router's history can my ISP still provide it to my parents?

How do I append a character to the end of every line in an Excel cell?

A Missing Symbol for This Logo

What is the difference between rolling more dice versus fewer dice?

How to play electric guitar and bass as a duet

Finding a logistic regression model which can achieve zero error on a training set training data for a binary classification problem with two features

Why is it that Bernie Sanders is always called a "socialist"?

How would an AI self awareness kill switch work?

Eww, those bytes are gross

Why did Luke use his left hand to shoot?

In Linux what happens if 1000 files in a directory are moved to another location while another 300 files were added to the source directory?

Am I a Rude Number?

Can my spouse sponsor me for a UK visa if I am unemployed?

Citing paywalled articles accessed via illegal web sharing

Play Zip, Zap, Zop

Why are the books in the Game of Thrones citadel library shelved spine inwards?



Downloading GitHub Archive files using wget: has {} for iteration


Problems downloading files with WGETDownloading all files from a remote HTTP folder using wgetwget terminates unexpectedlyDownloading an (apache) online directory locally using cli (wget …)Wget/bash downloading certain DIVUsing wget does not give same output as browserwget alternative to downloading fileswget error 400 CentOSDownloading with wget using referer + cookieNot able to download Oracle's jdk-8u181 package using wget behind an HTTP proxy













0















If I copy/paste this in the command line, I can download ALL GitHub archive files for 24 hours on January 1st 2015:



wget http://data.gharchive.org/2015-01-01-{0..23}.json.gz


I want to be able to do this for multiple days, using a script. For example, in my .sh file:



#!/bin/bash

while read line
do
wget $line
done < download_github_files.txt


And in download_github_files.txt, I have:



http://data.gharchive.org/2015-01-01-{0..23}.json.gz
http://data.gharchive.org/2015-01-02-{0..23}.json.gz
http://data.gharchive.org/2015-01-03-{0..23}.json.gz


Unfortunately, when I run the .sh file, I get three printouts of:



--2019-02-27 19:00:28--  http://data.gharchive.org/2015-01-01-'%7B'0..23'%7D'.json.gz
Resolving data.gharchive.org (data.gharchive.org)...
Connecting to data.gharchive.org (data.gharchive.org)...
connected.
HTTP request sent, awaiting response... 404 Not Found
2019-02-27 19:00:28 ERROR 404: Not Found.


The problem is clearly with {. Escaping that character doesn't help, because it's meant to be an iterator from what I can tell.



How can I download these files using wget?










share|improve this question









New contributor




stats_sorceress is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

























    0















    If I copy/paste this in the command line, I can download ALL GitHub archive files for 24 hours on January 1st 2015:



    wget http://data.gharchive.org/2015-01-01-{0..23}.json.gz


    I want to be able to do this for multiple days, using a script. For example, in my .sh file:



    #!/bin/bash

    while read line
    do
    wget $line
    done < download_github_files.txt


    And in download_github_files.txt, I have:



    http://data.gharchive.org/2015-01-01-{0..23}.json.gz
    http://data.gharchive.org/2015-01-02-{0..23}.json.gz
    http://data.gharchive.org/2015-01-03-{0..23}.json.gz


    Unfortunately, when I run the .sh file, I get three printouts of:



    --2019-02-27 19:00:28--  http://data.gharchive.org/2015-01-01-'%7B'0..23'%7D'.json.gz
    Resolving data.gharchive.org (data.gharchive.org)...
    Connecting to data.gharchive.org (data.gharchive.org)...
    connected.
    HTTP request sent, awaiting response... 404 Not Found
    2019-02-27 19:00:28 ERROR 404: Not Found.


    The problem is clearly with {. Escaping that character doesn't help, because it's meant to be an iterator from what I can tell.



    How can I download these files using wget?










    share|improve this question









    New contributor




    stats_sorceress is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.























      0












      0








      0








      If I copy/paste this in the command line, I can download ALL GitHub archive files for 24 hours on January 1st 2015:



      wget http://data.gharchive.org/2015-01-01-{0..23}.json.gz


      I want to be able to do this for multiple days, using a script. For example, in my .sh file:



      #!/bin/bash

      while read line
      do
      wget $line
      done < download_github_files.txt


      And in download_github_files.txt, I have:



      http://data.gharchive.org/2015-01-01-{0..23}.json.gz
      http://data.gharchive.org/2015-01-02-{0..23}.json.gz
      http://data.gharchive.org/2015-01-03-{0..23}.json.gz


      Unfortunately, when I run the .sh file, I get three printouts of:



      --2019-02-27 19:00:28--  http://data.gharchive.org/2015-01-01-'%7B'0..23'%7D'.json.gz
      Resolving data.gharchive.org (data.gharchive.org)...
      Connecting to data.gharchive.org (data.gharchive.org)...
      connected.
      HTTP request sent, awaiting response... 404 Not Found
      2019-02-27 19:00:28 ERROR 404: Not Found.


      The problem is clearly with {. Escaping that character doesn't help, because it's meant to be an iterator from what I can tell.



      How can I download these files using wget?










      share|improve this question









      New contributor




      stats_sorceress is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.












      If I copy/paste this in the command line, I can download ALL GitHub archive files for 24 hours on January 1st 2015:



      wget http://data.gharchive.org/2015-01-01-{0..23}.json.gz


      I want to be able to do this for multiple days, using a script. For example, in my .sh file:



      #!/bin/bash

      while read line
      do
      wget $line
      done < download_github_files.txt


      And in download_github_files.txt, I have:



      http://data.gharchive.org/2015-01-01-{0..23}.json.gz
      http://data.gharchive.org/2015-01-02-{0..23}.json.gz
      http://data.gharchive.org/2015-01-03-{0..23}.json.gz


      Unfortunately, when I run the .sh file, I get three printouts of:



      --2019-02-27 19:00:28--  http://data.gharchive.org/2015-01-01-'%7B'0..23'%7D'.json.gz
      Resolving data.gharchive.org (data.gharchive.org)...
      Connecting to data.gharchive.org (data.gharchive.org)...
      connected.
      HTTP request sent, awaiting response... 404 Not Found
      2019-02-27 19:00:28 ERROR 404: Not Found.


      The problem is clearly with {. Escaping that character doesn't help, because it's meant to be an iterator from what I can tell.



      How can I download these files using wget?







      command-line bash wget






      share|improve this question









      New contributor




      stats_sorceress is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.











      share|improve this question









      New contributor




      stats_sorceress is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      share|improve this question




      share|improve this question








      edited 2 mins ago









      JakeGould

      31.5k1096138




      31.5k1096138






      New contributor




      stats_sorceress is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      asked 19 mins ago









      stats_sorceressstats_sorceress

      1




      1




      New contributor




      stats_sorceress is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.





      New contributor





      stats_sorceress is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






      stats_sorceress is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






















          0






          active

          oldest

          votes











          Your Answer








          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "3"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });






          stats_sorceress is a new contributor. Be nice, and check out our Code of Conduct.










          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1410030%2fdownloading-github-archive-files-using-wget-has-for-iteration%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          stats_sorceress is a new contributor. Be nice, and check out our Code of Conduct.










          draft saved

          draft discarded


















          stats_sorceress is a new contributor. Be nice, and check out our Code of Conduct.













          stats_sorceress is a new contributor. Be nice, and check out our Code of Conduct.












          stats_sorceress is a new contributor. Be nice, and check out our Code of Conduct.
















          Thanks for contributing an answer to Super User!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1410030%2fdownloading-github-archive-files-using-wget-has-for-iteration%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Couldn't open a raw socket. Error: Permission denied (13) (nmap)Is it possible to run networking commands...

          VNC viewer RFB protocol error: bad desktop size 0x0I Cannot Type the Key 'd' (lowercase) in VNC Viewer...

          Why not use the yoke to control yaw, as well as pitch and roll? Announcing the arrival of...