Downloading GitHub Archive files using wget: has {} for iterationProblems downloading files with...
How can my powered armor quickly replace its ceramic plates?
Why am I able to open Wireshark in macOS without root privileges?
A starship is travelling at 0.9c and collides with a small rock. Will it leave a clean hole through, or will more happen?
What would be the rarity of this magic item(s)?
False written accusations not made public - is there law to cover this?
Early credit roll before the end of the film
Why publish a research paper when a blog post or a lecture slide can have more citation count than a journal paper?
Is it possible to grant users sftp access without shell access? If yes, how is it implemented?
If I delete my router's history can my ISP still provide it to my parents?
How do I append a character to the end of every line in an Excel cell?
A Missing Symbol for This Logo
What is the difference between rolling more dice versus fewer dice?
How to play electric guitar and bass as a duet
Finding a logistic regression model which can achieve zero error on a training set training data for a binary classification problem with two features
Why is it that Bernie Sanders is always called a "socialist"?
How would an AI self awareness kill switch work?
Eww, those bytes are gross
Why did Luke use his left hand to shoot?
In Linux what happens if 1000 files in a directory are moved to another location while another 300 files were added to the source directory?
Am I a Rude Number?
Can my spouse sponsor me for a UK visa if I am unemployed?
Citing paywalled articles accessed via illegal web sharing
Play Zip, Zap, Zop
Why are the books in the Game of Thrones citadel library shelved spine inwards?
Downloading GitHub Archive files using wget: has {} for iteration
Problems downloading files with WGETDownloading all files from a remote HTTP folder using wgetwget terminates unexpectedlyDownloading an (apache) online directory locally using cli (wget …)Wget/bash downloading certain DIVUsing wget does not give same output as browserwget alternative to downloading fileswget error 400 CentOSDownloading with wget using referer + cookieNot able to download Oracle's jdk-8u181 package using wget behind an HTTP proxy
If I copy/paste this in the command line, I can download ALL GitHub archive files for 24 hours on January 1st 2015:
wget http://data.gharchive.org/2015-01-01-{0..23}.json.gz
I want to be able to do this for multiple days, using a script. For example, in my .sh file:
#!/bin/bash
while read line
do
wget $line
done < download_github_files.txt
And in download_github_files.txt
, I have:
http://data.gharchive.org/2015-01-01-{0..23}.json.gz
http://data.gharchive.org/2015-01-02-{0..23}.json.gz
http://data.gharchive.org/2015-01-03-{0..23}.json.gz
Unfortunately, when I run the .sh file, I get three printouts of:
--2019-02-27 19:00:28-- http://data.gharchive.org/2015-01-01-'%7B'0..23'%7D'.json.gz
Resolving data.gharchive.org (data.gharchive.org)...
Connecting to data.gharchive.org (data.gharchive.org)...
connected.
HTTP request sent, awaiting response... 404 Not Found
2019-02-27 19:00:28 ERROR 404: Not Found.
The problem is clearly with {
. Escaping that character doesn't help, because it's meant to be an iterator from what I can tell.
How can I download these files using wget
?
command-line bash wget
New contributor
add a comment |
If I copy/paste this in the command line, I can download ALL GitHub archive files for 24 hours on January 1st 2015:
wget http://data.gharchive.org/2015-01-01-{0..23}.json.gz
I want to be able to do this for multiple days, using a script. For example, in my .sh file:
#!/bin/bash
while read line
do
wget $line
done < download_github_files.txt
And in download_github_files.txt
, I have:
http://data.gharchive.org/2015-01-01-{0..23}.json.gz
http://data.gharchive.org/2015-01-02-{0..23}.json.gz
http://data.gharchive.org/2015-01-03-{0..23}.json.gz
Unfortunately, when I run the .sh file, I get three printouts of:
--2019-02-27 19:00:28-- http://data.gharchive.org/2015-01-01-'%7B'0..23'%7D'.json.gz
Resolving data.gharchive.org (data.gharchive.org)...
Connecting to data.gharchive.org (data.gharchive.org)...
connected.
HTTP request sent, awaiting response... 404 Not Found
2019-02-27 19:00:28 ERROR 404: Not Found.
The problem is clearly with {
. Escaping that character doesn't help, because it's meant to be an iterator from what I can tell.
How can I download these files using wget
?
command-line bash wget
New contributor
add a comment |
If I copy/paste this in the command line, I can download ALL GitHub archive files for 24 hours on January 1st 2015:
wget http://data.gharchive.org/2015-01-01-{0..23}.json.gz
I want to be able to do this for multiple days, using a script. For example, in my .sh file:
#!/bin/bash
while read line
do
wget $line
done < download_github_files.txt
And in download_github_files.txt
, I have:
http://data.gharchive.org/2015-01-01-{0..23}.json.gz
http://data.gharchive.org/2015-01-02-{0..23}.json.gz
http://data.gharchive.org/2015-01-03-{0..23}.json.gz
Unfortunately, when I run the .sh file, I get three printouts of:
--2019-02-27 19:00:28-- http://data.gharchive.org/2015-01-01-'%7B'0..23'%7D'.json.gz
Resolving data.gharchive.org (data.gharchive.org)...
Connecting to data.gharchive.org (data.gharchive.org)...
connected.
HTTP request sent, awaiting response... 404 Not Found
2019-02-27 19:00:28 ERROR 404: Not Found.
The problem is clearly with {
. Escaping that character doesn't help, because it's meant to be an iterator from what I can tell.
How can I download these files using wget
?
command-line bash wget
New contributor
If I copy/paste this in the command line, I can download ALL GitHub archive files for 24 hours on January 1st 2015:
wget http://data.gharchive.org/2015-01-01-{0..23}.json.gz
I want to be able to do this for multiple days, using a script. For example, in my .sh file:
#!/bin/bash
while read line
do
wget $line
done < download_github_files.txt
And in download_github_files.txt
, I have:
http://data.gharchive.org/2015-01-01-{0..23}.json.gz
http://data.gharchive.org/2015-01-02-{0..23}.json.gz
http://data.gharchive.org/2015-01-03-{0..23}.json.gz
Unfortunately, when I run the .sh file, I get three printouts of:
--2019-02-27 19:00:28-- http://data.gharchive.org/2015-01-01-'%7B'0..23'%7D'.json.gz
Resolving data.gharchive.org (data.gharchive.org)...
Connecting to data.gharchive.org (data.gharchive.org)...
connected.
HTTP request sent, awaiting response... 404 Not Found
2019-02-27 19:00:28 ERROR 404: Not Found.
The problem is clearly with {
. Escaping that character doesn't help, because it's meant to be an iterator from what I can tell.
How can I download these files using wget
?
command-line bash wget
command-line bash wget
New contributor
New contributor
edited 2 mins ago
JakeGould
31.5k1096138
31.5k1096138
New contributor
asked 19 mins ago
stats_sorceressstats_sorceress
1
1
New contributor
New contributor
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
stats_sorceress is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1410030%2fdownloading-github-archive-files-using-wget-has-for-iteration%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
stats_sorceress is a new contributor. Be nice, and check out our Code of Conduct.
stats_sorceress is a new contributor. Be nice, and check out our Code of Conduct.
stats_sorceress is a new contributor. Be nice, and check out our Code of Conduct.
stats_sorceress is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1410030%2fdownloading-github-archive-files-using-wget-has-for-iteration%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown