Quantcast
Channel: Adobe Community: Message List
Viewing all articles
Browse latest Browse all 77806

Re: Moving between NAS and local drive

$
0
0

How is your local storage connected to the computer and what type of storage is it?  For example, a USB2 standard hard drive will never match the bandwidth that you'll see from a Thunderbolt2 SSD drive.

No problems in that department!  First an anecdote: after a trip in 2008 when I started using a dSLR w/RAW files, my wife's computer couldn't view them (over the network) very well, so although she was just happy with what she had up to that point for what she used, I upgraded the motherboard/CPU with one that had gigabit Ethernet an more RAM.  As expected, gigabit was the next best thing to being there, compared to the drive's sustained speed of 100MB/s over SATA.

 

USB drives and enclosures, on the other hand, almost always had a hard limit of 20MB/s no matter what it said on the box about USB specs or what drive was used.  Since that's not in fact universal, I think it's a limitation of the inexpensive chip used in such devices.

 

(BTW, I'm using a PC, not a Mac, so I can attach more drives directly to SATA on the motherboard, and don't have Thunderbolt)

 

My current NAS is connected via gigabit Ethernet, with it and our desktops connected to the same switch.

 

The NAS is powered by a Xeon Haswell E3-1245 on a Supermicro X10SL7-F, with 16GB of ECC unbuffered DRAM.  It contains five 4TB drives, with 3-of-5 redundancy (Called Z2, same ideas as RAID6).  It is running firmware FreeNAS-9.2.1.8-RELEASE-x64 (e625626) if that matters.

 

Doing a file-copy, "cold" after just booting up, took 19 seconds to copy one 2G2B WAV file.  Another file took 27 seconds for for 2.861 G10Bytes.  That's 105MB/s (848Mbits application layer PDU) with compares favorably to the approximate 111¼ MB/s (890 Mbits) bandwidth.

 

That's available independently in each direction, so reading and writing about the same size result back to the same directory would have the same speed, unlike the case for local (mechanical) drives where the seeking takes a toll, as I noticed earlier when filtering, encoding, etc. using command line tools.

 

I want the files stored in the ZFS file system (not in a container file mounted as a block device re SAN) because of the robustness and snapshot capabilities, and any issues with the local computer won't damage the file system itself which is controlled by its own CPU.

 

The access speed is not a  problem.  Drives accessed over network are not cached by the OS, which is a huge performance hit for applications that re-read the same small set of files (e.g. C++ compiler) or jump around and re-read a portion that it already just read (I'm not sure if that's always true or varies significantly depending on the mode and sharing flags used to open the file; without writing dedicated test software I can't tell for sure if a program is smart enough to not re-read something sometimes, or the OS is; but that OS caching only persists for the duration of the specific file handle).

 

The observation that the IO activity was very low and the CPU was high makes me think the software has an inherent problem with the file being on a network mount, for this specific function.  Lightroom, Photoshop, and editing I've done in Premiere thus far have not been a problem.

 

NAS is not the right type of storage for video editing.  If you really want to work across a network, you'd want a SAN.

 

Premiere caches rendered data on a local RAM disk, and that's what I see when I scrub and whatnot.  So I've not seen for myself what the concern is that leads to; what (else) can happen?  (Actually, I have seen the QuickTime library (used by Resolve) does not handle networks for some as-yet-unknown reason.)


Viewing all articles
Browse latest Browse all 77806

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>