You'll have to complete two quests in order to get this key. The first is "Pest Control" and the second is "Gift Wood for Driftwood". After finishing "Pest Control", the game needs some time get itself in order, so go do some more missions, or search for other keys for a while.

Once these are finished, cross over to the island camp, using the new bridge to reach the switch. When you are close enough to this switch, shoot it or strike it and watch it flee! If you haven't given the Carpenter enough time (once again), you will eventually come to a spot where you won't be able to hit the switch (across the channel). Go have some more orgies or whatever until the game catches up.

Keep chasing the annoying bugger, hitting or shooting it when you can, until it stops at the top of the mountain. The key is at the end of this journey, through a gate.