You two talking in another language and laughing won't exactly put your new office-mate at ease, you know?

As for not wanting to shake hands... Well, it's not a custom or tradition that I'm familiar with... But overall pretty harmless. I do want to point out that there is no need to "respect" the things about other people that make no sense to you. Simply tolerate and deal with them. That would be my approach, at any rate.