Integration with automake
First copy the Makefile.am from the examples sub directory of the gtkdoc-sources to your project's API documentation directory ( ./docs/reference/<package>). A local copy should be available under e.g. /usr/share/doc/gtk-doc-tools/examples/Makefile.am. If you have multiple doc-packages repeat this for each one.
The next step is to edit the settings inside the Makefile.am. All the settings have a comment above that describes their purpose. Most settings are extra flags passed to the respective tools. Every tool has a variable of the form <TOOLNAME>_OPTIONS. All the tools support --help to list the supported parameters.