I do think running the code would be a tiny bit faster, even if it's merely seconds either way. Opening a python REPL and pasting that would take around 5 seconds in my case. Running the code in my head would take roughly the same at first, but then if it's in an interview I'd take the time to double check. And then check a few more times because I'd expect some kind of trick here.
Considering there's no (explicit) instruction forbidding or discouraging it, I'd consider the REPL solution to be perfectly valid. In fact some interview tests specifically look for this kind of problem solving.
I get it still, I'd expect some valuable signal from this test. Candidates who execute this code are likely to do so because they really want to avoid running the code in their head, not just because it's more straightforward, and that's probably a bad sign. And pasting that into an LLM instead of a REPL would be a massive red flag.
I just don't think answering "-11" here is a signal strong enough to disqualify candidates on its own.
If you're looking for junior-ish python devs, I'd expect a good chunk of the better ones to have a python repl open and ready just as a matter of habit.
So for them, yes, it would clearly be faster to run the code than to work through it manually.
What you're doing here is selecting for candidates who are less comfortable with using the tools that they'd be expected to use every day in the role you're hiring for. It's likely to provide a negative signal.
So I wouldn't go so far as to say that I'd fire someone for copying and pasting code, but it's definitely part of my company's culture that copying and pasting code off of a website, and especially executing it, is something heavily discouraged to the point that it doesn't really happen at my job.
I'm perfectly happy to use Stack Overflow and other resources/tutorials, blog posts etc... to find solutions to problems, but just instinctively I would never think to copy and paste a solution from these sites and incorporate it into my codebase and I sure as heck wouldn't think to execute code from some untrusted site I happened to come across.
But this may also be a consequence of the domain I work in where we take security very seriously.
You can tell how safe a code snippet is from reading it.
Like, there's no way you're going to copy a 20 line algorithm from stack overflow on balancing a red-black tree and have it encrypt your harddrive.
Obviously you still need to test the code to make sure it works and understand what it's doing, but there is very little security risk here. Just look up the functions youre using and understand the code and you're fine.
