Please read Part 1 first.
Part of me wants to see if this documentary also covers the hard drive firmware virus. The geek in me wants to tip my hat to the people who created that thing because there is almost no way to purge that. Hard drives never used to have flashable firmware. I guess they do now.
If your eyes glaze over now I will understand, but, you really need to understand a bit about this to understand just how at-risk you really are in today’s world. I’m old enough to remember booting the DOS DEBUG utility to format MFM hard drives for DOS based PCs. I even remember the run-it-overnight COMPSURF utility for Netware servers.
Back in the day all PC and most larger computer hard drives were formatted via tracks and sectors. IBM used to use terms like tracks and cylinders for allocation. This wikipedia article contains an image which most people think of when they think of sectors and tracks on a hard drive. The image is incorrect. Sectors were a fixed size of 512 bytes. Due to the nature of a disk platter (think of a vinyl LP and its groves) the tracks increased in size as you moved out from the center. this meant that as you moved to the outer tracks very little of the drive media was actually used because each sector had to start in the same spot as the corresponding sector in the inner most track.
There were various utilities and viruses which could force MFM drives to write into the space between the sectors. This was the IT industries first foray into “viruses which couldn’t be purged.” Normal formatting methods would not overwrite those spaces so the virus stayed in place waiting for head drift to find it again. Read/Write drive heads were on physical arms back then, much like the needle arm of a record player. Eventually they would develop a bit of play in their movements called drift. When this started happening you usually had to low level format the drive again to make it usable for a while longer. Back in the days of MFM your computer software directly addressed head and track to identify the sector which needed to be read or written. In short the computer and DOS completely controlled the disk drive, hence the name DOS for Disk Operating System.
Various other technologies for mapping drives eventually came about. In large part this was due to the limits of first 8-bit then 16-bit CPU contained in PCs. The maximum unsigned 8-bit value is the magic number 255. During the early days of IDE disk drives they used to play games with the computer BIOS presenting 255 heads and 255 tracks/cylinders. This yielded a maximum storage space of:
255*255*512 = 33292800 bytes
33292800 / 1024 = 32512.5 K
32512.5 / 1024 = 31.750488281 Meg
That is correct, your largest hard drive using that trick was roughly 30 Meg. No where near large enough to store any of the current PC operating systems let alone video. Some of you have probably created document files larger than that for either work or school.
My hardware skills start to slack off at this point. When we went to 16-bit computers the maximum value a register could hold was 65,535. Many different addressing schemes came into play. I’ve kind of blurred the definition of MFM and LBA (Logical Block Addressing) with my example. If some wish to call me on it fine, but the barrier was real. You can click the link for LBA if you wish to dig into it further.
What had to and eventually did happen was moving of control out to either the hard drive or controller. Other tricks included adding support for cluster size which is also called default allocation size.
At any rate, we have yet another example of people lunging ahead without actually thinking anything through, just like they didn’t think anything through creating the Internet. Yes there was an election campaign based on “creating a global village” but the idiots failed to create the global village council first. That is why the Internet is currently rife with crime and fraud. If you think the regular Internet is rampant with crime just read up on or (gasp!) visit “the Dark Web.” You’ve all heard of Silk Road. It may be the most famous but it is far from the only one.
I have not read “The Dark Net” but I did hear the author interview on NPR. If you have any interest in that topic I recommend finding a copy of that book. The interview was very informative and the author spent about a year surfing through the Dark Net. What is important for this discussion is the amount of malicious source code available in that cesspit.
By moving control of storage farther out of the operating system the hardware and OS world created an insidious point of failure. In the old days you could low level format a hard drive to remove most viruses. That was your worst case. Old drives were mechanical idiots. Today’s storage devices have what amounts to a communications specification. Your operating system requests something and it gives the something to you. Exactly where it is and how it gets it you don’t really know. Yes, many of you will have heard of partitions being formatted in NTFS, ext4, etc. but, exactly where the clusters and sectors are we do not know. With those little SD cards or physically larger SSD (Solid State Drive) disk drives, there aren’t any moving parts. There is just firmware which makes this type of storage appear as a disk drive to whatever operating system it is used on.
Today’s operating systems and virus protection software weren’t looking for viruses loaded into the firmware of the drive nor were they looking for software which tried to update drive firmware. You cannot low level format and restore from backup your way out of this one. First you have to find valid firmware, then you have to figure out how to update it on your drive. This, of course, assumes the virus isn’t smart enough to hook into the portion of the API/communications scheme for flashing firmware and store itself into what you are about to load before letting you reload the flash.
How did this happen? Simple. We used to have to use something called an PROM which got “burned” with firmware at the factory and could never be changed. The only way to correct problems in the field was to ship out an updated drive and get the old drive shipped back. It was expensive and time consuming. Then we moved to EPROM which could be erased and reprogrammed then a rash of other technologies which could be field updated. In seeking convenience with cost reduction risk was raised. With the advent of disk drive firmware viruses the chickens have come home to roost. It is now theoretically possible to have a firmware virus which hides the existence of other viruses on the hard drive by adding their sectors to the bad sector list yet allowing them to execute.
USB ports are wonderful, but I fear they will be the next insidious point of penetration. Some level of intelligence has to exist on the devices we plug into any given USB port as does some level of intelligence within the USB controller. If any of these are flashable it is only a matter of time before some intelligence agency or the denizens of the Dark Web figure out how to exploit that.
Without a global village council the global village duth run amok.