It's kind of cute when Casper is so flustered, but Ryoji also really doesn't want to make him feel bad. Which is certainly what the initial reaction looks like, so he pulls back, concern all over his face as he examines Casper's, trying to figure out where he went wrong, exactly.

Casper explaining, well... it doesn't actually explain a whole lot to Ryoji. But he offers Casper a gentle smile and a tilt of his head, not moving further away than he already had. "I'm sorry, I didn't mean to freak you out... but... what do you mean by you 'know' I'm just being friendly?"

Yeah he is definitely still flirting.