Averaging in gamma versus linear light
A JPG does not store light, it stores numbers chosen to suit the eye. Average those numbers and the result is darker than the light they stand for — which is why a blurred string of fairy lights turns into a grey smear.
The short answer
A pixel value of 128 is not half the light of 255; it is about a fifth of it. Photo files are gamma-encoded so that the 256 available steps are spent where the eye notices differences, in the darks. That encoding is perfect for storing a picture and wrong for mixing one: the average of a black pixel and a white pixel ought to be the grey that emits half the light, which is stored as about 188, not 128.