Your choices make a game casual or not. It's up to you to stop playing a game so you can do something else. If you set a time limit and then you get to that limit and say to yourself 'Well, times up, so I'll only play for another hour.' and you end up not walking away for another four hours then you've chosen to not play casually.
Set a time limit and shut the game down at the end of that time. That is how you play casually. Telling yourself 'just one more quest' or 'just 40 more Gorflaxen ulnas' is not playing casually.