Ever wondered why your s3-file not found isn't working when you are caching resources with CloudFront? Well that is because the static web hosting function and it's 404 are not part of the s3-bucket access.
To enable the file-not-found lambda functions etc, you have to specify the website hostname instead of the s3 bucket. e.g., static.example.eu.s3-website-eu-central-1.amazonaws.com
instead of static.example.eu.s3.amazonaws.com
The documentation here does specify it very "clearly"...
4: On the Create Distribution page, in the Origin Settings section, for Origin Domain Name, type the Amazon S3 static website hosting endpoint for your bucket. For example, example.com.s3-website-us-east-1.amazonaws.com.
The problem is, you will definitely read over that. So for all those that couldn't figure it out, this is usually the problem!