x2x

x2x allows the keyboard, mouse on one X display to be used to control another X display. It also shares X clipboards between the displays.

Install it on ubuntu is as easy as:

sudo apt-get install x2x

start it up on the server:

ssh -XC laptop x2x -east -to :0.0

Voila, now your keyboard,mouse and clipboard is shared with another x server...