County of Fortune move settlement: select the settlement, click the "Move settlement to a different location" icon in the top-left corner of the settlement menu, and move it to the new site. Moving is supported, it costs nothing, and there is no penalty attached to it.
County of Fortune move settlement: how to relocate a village
Select the settlement, click the "Move settlement to a different location" icon in the top-left corner of the settlement menu, and drag it to the site you want. The developers added the feature specifically so that a misclick, a badly understood piece of terrain, or a late realisation that the settlement would work better nearer a resource is not punished. It was the most requested feature of the June Steam Next Fest demo, and it is present in the current demo and in Early Access.
If you are still choosing sites, where to build settlements is the page to read first.
Why you cannot carry a village across the map
A settlement has to stay near the place where it was originally founded, no matter how many times it has already been moved. Settlement movement is not meant to be a way to relocate villages freely across the entire map, because a settlement must always remain near its founding place.
That makes relocation a correction of an early decision rather than a way to shuffle your county around after the fact. A settlement always stays near the place it was founded, so plan the site before you commit to it.
When moving is not the answer
Moving fixes a badly chosen site. It does not fix a settlement whose buildings are wrong for the region — for that you pull the buildings out and put different ones in, which changing production covers.
And if what you really want is for the village not to exist at all, there is no way to remove it: deleting a settlement explains what you can do instead.