A microprocessor needs a component that generates clock signals which basically control when and to which beat everything happens.
The ATmega MCUs have a built-in "RC oscillator" but you can also use an external resonator or crystal. The easier the solution, the less precise clock signals will be generated (ain't no such thing as a free lunch :-).
The easiest solution is to use the built-in RC oscillator, the second easiest is to use an external resonator which doesn't require any other components and the lastly you can use an external crystal and a pair of "load capacitors".
If you don't need precise timing the internal RC oscillator is completely adequate but if you use serial communication or other tasks where precise timing is required, you should use a crystal or a resonator.
Read this excellent guide to choosing a clock source: Why you need a clock source - an introduction to choosing and using clock sources.
When using a crystal two capacitors need to be connected from the crystal to GND. Usually a "load capacitance" is specified in the crystal's datasheet but this is total load capacitance so you need a different capacitance for the two parallel capacitors. To find the required capacitance (rLC), the following formula can be used:
rLC = 2 x (LC-PC)
where LC is the specificed total load capacitance and PC is parasitic capacitance (from nearby wires and circuit board etc.) which is usually between 7 and 10 pF.
Another useful formula is this one which is used to calculate the load capacitance when using two capacitors of a capacitance of rLC:
LC = rLC^2 / 2rLC + PC
An example: this 16.000 MHz crystal requires a load capacitance of 30pF. Using the first formula yields using a rLC of 40 pF:
rLC = 2 x (30pF - 10pF) => rLC = 40pF
However, if you don't have all exact size capacitors in stock (I know I don't) you can choose the nearest and use the second formular to check what the load capacitance will be. So for two capacitors of 39 pF (again using a value of 10pF for PF) we get:
LC = 39 pF x 39 pF / 2 x 39 pF + 10 pF => LC = 1521 pF / 78 pF + 10 pF => LC = 29.5 pF
which is close enough.