Painting with light. Exactly. That’s exactly the end goal of the lighting pipeline, and the creative lighting process.
The thing is, in order to effectively do that, you need shaders that ALL behave in a physically plausible, standardized way towards that light. Without that, the most creative lighting in the world is meaningless, because none of the objects you’re directing that light towards will appear to sit together in the same world at all.
You’ve described a film set totally right. The people working on those sets pull out a ton of tricks and use all sorts of techniques to get the look they want. However, they are working with REAL materials, and REAL photons. No matter how absurd the lighting set up and cameras end up tricked out, there is never a concern that the prop artist who made the milk jug has it reflecting 2x more energy than guy who made the cookie jar.
To me as an artist, my dream software package would give me the most physically accurate surface model that can be computed in a reasonable time, and it would ship with a full set of digitally recreated practical lights, along with fully recreated camera and lens systems for all of the common bodies and lenses that I’d want to shoot on.
I would then be doing the closest possible thing to placing real lights among real objects and shooting with a real camera. I’d have the perfect balance of maximum creativity, while also having maximum realism. All of the materials and surfaces would behave like they do in the real world, and my job would be all about lighting it to look as good as possible.
This discussion isn’t about lighting though, it’s about shaders…and at the end of the day, the default state of a shader HAS TO BE that it conserves energy and is a physically plausible surface model. You’ve always got the option of breaking that, adding things together, whatever you see fit…but the baseline absolutely must be physical plausibility.
There isn’t some conspiracy going on between studios and rendering packages. It just happens to be the most globally agreed upon version of what amounts to “best practices”, and happens to be the working methodology that produces the most convincing and most beautiful results.
If I can load up a scene full of assets, and all those assets share sane values that were arrived at with reference sheets for reflectance, roughness, and diffuse values…if all those assets sit in the same world, my job as a lighter becomes more about ACTUALLY lighting. With fully unlocked shading models, my job as a lighter is more about solving and fixing problems, and hoping to still get a few iterations before my time is up with the shot/project.
Without physically plausible assets, you have a much lower chance of success with lighting artistry, and with creatively making a beautiful shot.
Cheating lights all over the place is what it’s all about on a film set. Cheating materials though? You literally can’t do that.
That’s the end goal of energy conserving surface models, and more recent microfacet surface models. They’re meant to give you something closer to a real world lighting workflow, where the materials and surfaces are all physically correct, and it’s up to you to light them nicely.
And if the point was to “keep feeding mediocre shots to the public”, companies like Weta, ILM, Pixar, etc., wouldn’t be investing dozens of millions per year into research that furthers the art of shading, lighting, and rendering the way that they do. Honestly that’s a really ignorant thought. Whether you like the projects or not, these studios are putting some of the most gorgeous imagery onto the screen that has ever been created.
(plus you can always just add things together anyway and not adhere to conservation rules, so I don’t see the big deal at all regardless of how you want to work)

)