There is a C interface for linux syscalls defined in libc and described in man pages. There are also resources like this https://www.chromium.org/chromium-os/developer-library/reference/linux-constants/syscalls/ that describe syscall numbers, argument order, etc.

However, i cannot find a detailed documentation on how to use syscalls in assembly. In most cases this is not a problem as you can just use C interface as a reference, however, there is a small set syscalls like mmap that encode return values in a weird way. mmap in particular seems to return NULL - errno_code on error, but i'm not entirely sure of this. Is there a good resource that describes all such fine-grained details? (besides the kernel code ofc)

PS: I'm primarily interested in linux abi for now, but if there are resources that comprehensively describe windows abi, those would be also nice to know.