Tjyylli

Is it a Cyclops number? "Nobody" knows!

Should we avoid writing fiction about historical events without extensive research?

Why is the electrolytic capacitor not polarity sensitive?

“I had a flat in the centre of town, but I didn’t like living there, so …”

Align equations with text before one of them

Convert an array of objects to array of the objects' values

Does the in-code argument passing conventions used on PDP-11's have a name?

What does it mean when I add a new variable to my linear model and the R^2 stays the same?

Does the US political system, in principle, allow for a no-party system?

What can I do if someone tampers with my SSH public key?

Ultrafilters as a double dual

A bug in Excel? Conditional formatting for marking duplicates also highlights unique value

What is Tony Stark injecting into himself in Iron Man 3?

Giving a talk in my old university, how prominently should I tell students my salary?

School performs periodic password audits. Is my password compromised?

Why is there an extra space when I type "ls" on the Desktop?

Is there such a thing in math the inverse of a sequence?

What is "desert glass" and what does it do to the PCs?

ESPP--any reason not to go all in?

Do natural melee weapons (from racial traits) trigger Improved Divine Smite?

How can I be pwned if I'm not registered on the compromised site?

Practical reasons to have both a large police force and bounty hunting network?

The (Easy) Road to Code

What does "rhumatis" mean?

How do I optimize Windows disk I/O for high-latency throughput?

wireless router - configuring for low-latency, high traffic environmenthigh latency on LANDisk-to-disk copying speedCatastrophic HDD performance degradationHow to benchmark hard disk?Windows 8 hard disk usage 100%Hard disk much slower when performing several long simultaneous readsadding Virtual Disk for performance (decreased latency) in VM EnvironmentHigh Latency acpi.sys & Wdf01000.sysPoor concurrent IO performance, how to trade latency for throughput?

1

1

Let's say there are 25 processes each which want to read and index a large (say, 1GB) file.
The hard drive may be capable of ~100MB/sec throughput, but the head movement and rotational latency involved in moving between such processes is significant so may reduce throughput to 5MB/sec, or just 5% of the conventional drive's throughput capability.

Were the hard drive to read a large amount of data (say, 32MB) or read whatever it can for a long time (say, 200ms) before moving onto the next process, file IO could approach the full throughput of the drive at the expense of latency.

Does Windows have such a facility for accomplishing this goal in cases where latency takes a back seat to throughput?

hard-drive performance latency windows-server-2012-r2 throughput

share|improve this question

asked Nov 1 '14 at 4:38

Charles BurnsCharles Burns

5701611

bumped to the homepage by Community♦ 20 hours ago

This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  check the tool DiskSpd: blogs.technet.com/b/josebda/archive/2014/10/13/…
  
  – magicandre1981
  Nov 1 '14 at 7:11
  

  
  
  
  

add a comment | 

1

1

Let's say there are 25 processes each which want to read and index a large (say, 1GB) file.
The hard drive may be capable of ~100MB/sec throughput, but the head movement and rotational latency involved in moving between such processes is significant so may reduce throughput to 5MB/sec, or just 5% of the conventional drive's throughput capability.

Were the hard drive to read a large amount of data (say, 32MB) or read whatever it can for a long time (say, 200ms) before moving onto the next process, file IO could approach the full throughput of the drive at the expense of latency.

Does Windows have such a facility for accomplishing this goal in cases where latency takes a back seat to throughput?

hard-drive performance latency windows-server-2012-r2 throughput

share|improve this question

asked Nov 1 '14 at 4:38

Charles BurnsCharles Burns

5701611

bumped to the homepage by Community♦ 20 hours ago

This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  check the tool DiskSpd: blogs.technet.com/b/josebda/archive/2014/10/13/…
  
  – magicandre1981
  Nov 1 '14 at 7:11
  

  
  
  
  

add a comment | 

Let's say there are 25 processes each which want to read and index a large (say, 1GB) file.
The hard drive may be capable of ~100MB/sec throughput, but the head movement and rotational latency involved in moving between such processes is significant so may reduce throughput to 5MB/sec, or just 5% of the conventional drive's throughput capability.

Were the hard drive to read a large amount of data (say, 32MB) or read whatever it can for a long time (say, 200ms) before moving onto the next process, file IO could approach the full throughput of the drive at the expense of latency.

Does Windows have such a facility for accomplishing this goal in cases where latency takes a back seat to throughput?

hard-drive performance latency windows-server-2012-r2 throughput

share|improve this question

asked Nov 1 '14 at 4:38

Charles BurnsCharles Burns

5701611

share|improve this question

asked Nov 1 '14 at 4:38

Charles BurnsCharles Burns

5701611

share|improve this question

asked Nov 1 '14 at 4:38

Charles BurnsCharles Burns

5701611

asked Nov 1 '14 at 4:38

Charles BurnsCharles Burns

5701611

asked Nov 1 '14 at 4:38

Charles BurnsCharles Burns

5701611

1 Answer
1

active

oldest

votes

0

This is what the NCQ protocol is for. A drive that supports NCQ looks at all of the incoming commands and reorders them to service them more efficiently (and essentially, reordering the commands is what you're asking about).

That said, I see in your example you have a disk queue depth of 27, so we're well past the help of even NCQ. This is a problem for RAID (possibly coupled with faster physical storage).

There are storage parameters that can be adjusted in Windows, particularly those related to caching behavior, but that's not going to help enough in your case.

share|improve this answer

edited Nov 2 '14 at 2:59

answered Nov 1 '14 at 12:28

Twisty ImpersonatorTwisty Impersonator

18.5k146699

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I think of NCQ as more for helping small, near random I/Os like database access. The drive involved does have NCQ and it probably helps a bit, bit I think this situation is more about tuning the Windows I/O scheduler. Linux allows me to choose I/O schedulers (deadline, CFQ...), but in fairness none of them would be much better than Windows' default in this case.
  
  – Charles Burns
  Nov 1 '14 at 14:28
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Additionally NCQ is in use by the drive whose performance is tested in the original question, still yielding less than 5% of the maximum throughput.
  
  – Charles Burns
  Apr 14 '16 at 19:02
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f834537%2fhow-do-i-optimize-windows-disk-i-o-for-high-latency-throughput%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

0

This is what the NCQ protocol is for. A drive that supports NCQ looks at all of the incoming commands and reorders them to service them more efficiently (and essentially, reordering the commands is what you're asking about).

That said, I see in your example you have a disk queue depth of 27, so we're well past the help of even NCQ. This is a problem for RAID (possibly coupled with faster physical storage).

There are storage parameters that can be adjusted in Windows, particularly those related to caching behavior, but that's not going to help enough in your case.

share|improve this answer

edited Nov 2 '14 at 2:59

answered Nov 1 '14 at 12:28

Twisty ImpersonatorTwisty Impersonator

18.5k146699

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I think of NCQ as more for helping small, near random I/Os like database access. The drive involved does have NCQ and it probably helps a bit, bit I think this situation is more about tuning the Windows I/O scheduler. Linux allows me to choose I/O schedulers (deadline, CFQ...), but in fairness none of them would be much better than Windows' default in this case.
  
  – Charles Burns
  Nov 1 '14 at 14:28
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Additionally NCQ is in use by the drive whose performance is tested in the original question, still yielding less than 5% of the maximum throughput.
  
  – Charles Burns
  Apr 14 '16 at 19:02
  

  
  
  
  

add a comment | 

0

This is what the NCQ protocol is for. A drive that supports NCQ looks at all of the incoming commands and reorders them to service them more efficiently (and essentially, reordering the commands is what you're asking about).

That said, I see in your example you have a disk queue depth of 27, so we're well past the help of even NCQ. This is a problem for RAID (possibly coupled with faster physical storage).

There are storage parameters that can be adjusted in Windows, particularly those related to caching behavior, but that's not going to help enough in your case.

share|improve this answer

edited Nov 2 '14 at 2:59

answered Nov 1 '14 at 12:28

Twisty ImpersonatorTwisty Impersonator

18.5k146699

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I think of NCQ as more for helping small, near random I/Os like database access. The drive involved does have NCQ and it probably helps a bit, bit I think this situation is more about tuning the Windows I/O scheduler. Linux allows me to choose I/O schedulers (deadline, CFQ...), but in fairness none of them would be much better than Windows' default in this case.
  
  – Charles Burns
  Nov 1 '14 at 14:28
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Additionally NCQ is in use by the drive whose performance is tested in the original question, still yielding less than 5% of the maximum throughput.
  
  – Charles Burns
  Apr 14 '16 at 19:02
  

  
  
  
  

add a comment | 

This is what the NCQ protocol is for. A drive that supports NCQ looks at all of the incoming commands and reorders them to service them more efficiently (and essentially, reordering the commands is what you're asking about).

That said, I see in your example you have a disk queue depth of 27, so we're well past the help of even NCQ. This is a problem for RAID (possibly coupled with faster physical storage).

There are storage parameters that can be adjusted in Windows, particularly those related to caching behavior, but that's not going to help enough in your case.

share|improve this answer

edited Nov 2 '14 at 2:59

answered Nov 1 '14 at 12:28

Twisty ImpersonatorTwisty Impersonator

18.5k146699

share|improve this answer

edited Nov 2 '14 at 2:59

answered Nov 1 '14 at 12:28

Twisty ImpersonatorTwisty Impersonator

18.5k146699

answered Nov 1 '14 at 12:28

Twisty ImpersonatorTwisty Impersonator

18.5k146699

answered Nov 1 '14 at 12:28

Twisty ImpersonatorTwisty Impersonator

18.5k146699