I first encountered finite differences in the book Numerical Analysis by Francis Scheid, and I think it is a fascinating concept. Not only it is useful in computer science; it is also useful in many ways for olympiad style mathematics. Its definition is simple: $f(x+1)-f(x)$, but the properties are endless. Here, I compiled some techniques used in problem solving that involve finite differences: Click here.