Incorrect 301 redirect to homepage when 404 would be expected

There is an issue which is probably related to URL canonicalisation that redirects to the homepage when a non-existent URL is requested, instead of returning 404. This is a common SEO anti-pattern but seems to be a bug in this case, not a "feature". So far disabling all plugins (including MU plugins bundled with the image) and changing to a default theme has not changed the behaviour.