The authority competent to grant or refuse leave to a Judge or revoke or curtail leave granted to a Judge shall be the Governor of the State in which the principal seat of the High Court is situate, after consultation with the Chief Justice of that High Court.