Okay, 'start' and 'restart' are clearly more comprehensible terms to the average person, but there is a richness of understanding in 'boot' and 'reboot' that has thereby been lost. Just as with the myth that 'bug' in reference to computer problems derived from an actual bug crawling into the innards of an ancient relay computer, thereby interrupting it, 'boot' was actually a metaphor for how a computer was started. ('Bug' was in use even by Thomas Edison, well before computers.)
The term came from another metaphor: 'Lifting himself up by his bootstraps'. This referred to someone who made good from humble beginnings, and reflected the inherent difficulties of succeeding despite not being equipped with the advantages that others enjoy. Consider, can you, by tugging on your bootstraps (shoe laces) lift yourself from the ground?
A similar conundrum is presented by computers. A computer is just a useless hunk of metal, plastic and silicon, unless something tells it what to do. In a sense, it is like the traditional concept of protoplasm. The idea was that living tissue was infused with some kind of living spirit that animated it. You could take, so it was thought, all the constituent parts of a dead human body, assemble them correctly, pump them up with healthy blood, but nothing would happen. The living spirit was missing. Mary Shelley had Dr Frankenstein insert the spark by capturing the power of lightening.
Protoplasm as a concept is entirely wrong, as it turns out (a year or two ago scientists made working polio virus from scratch, a pivotal step towards the synthetic creation of living organisms, and no lightening was needed to get it going). But it does neatly illustrate the problem with a computer.
The living spirit of a computer is a program. This is the 'machine code' that its CPU (a Pentium or whatever) reads to perform actions. But in order to read in a bit of machine code, a program already needs to be operating in the CPU.
In the very early days of computing this wasn't a problem because 'programming' the computer (the first model of ENIAC) was done by wiring the components together to make the program. This was before even the punch card. But since then computers have become multipurpose devices. They can do pretty much anything that can be designed into a program.
That is where booting comes in. To overcome the Catch 22 of needing to have a program in place in order to read a program so that it can operate, every computer CPU is indeed built like the old-fashioned valve computers. A 'boot' program is hardwired into it. Whenever power is applied to the CPU for the first time, it executes this program.
The program is tiny. Hardwired programming consumes a lot of space that is better preserved for running programs that actually do useful stuff, like calculating your finances or playing DVDs on your computer monitor. So the boot program's sole function is to tell the CPU to start loading in a program located at a particular position in the system's BIOS. BIOS stands for Basic In/Out System, and is a ROM (Read Only Memory) device. The BIOS generally doesn't actually have its main program where the CPU points to. Instead, that memory location consists of a small program which redirects the CPU to look somewhere else within the BIOS for the proper program.
This then is loaded, and really gets the whole show going. The new program has enough grunt to go and look at your computer's hard disk and load its Master Boot Record. This also is still a very small program, with a maximum length of 446 bytes on a IBM compatible personal computer. But it is big enough to the tell the computer where to find the real operating system and start it loading.
So what happens when you start your computer is a whole cascade of tiny computer programs, each with only a very few functions, and each with a primary function of calling the next program into operation, executing in turn. Finally the operating system proper gets started, and what happens then makes all that has gone before seem like child's play.