Cuda-gdb errors

I’ve been spending a lot of time modifying cuda applications to run in a GPU Simulator, GPGPU-Sim. Sometimes this can lead to errors that don’t make sense, especially if you’re not careful when using cuda-gdb.

Example: cudaMalloc was returning 0x9c inside cuda-gdb. For some reason, the recommended api error checking method wasn’t working, and weirdly, it didn’t trigger outside of cuda-gdb. Also, what the heck is 0x9c? To find out, grep through the cuda/include folder for the value. Make sure to add a comma after the value so you narrow the results. For my case, grep '201,' yields cuda.h: CUDA_ERROR_INVALID_CONTEXT = 201,

cudaMalloc doesn’t return this code, so something is happening at the driver level. Anyway, I found the error code information here

The relevant information is this: ..This can also be returned if a user mixes different API versions (i.e. 3010 context with 3020 API calls..

For my case, I was using the latest cuda-gdb, version 8.0. GPGPU-Sim (the non-devel version) requires a much older version of cuda. My application that I was debugging was compiled with cuda 3.1. For some reason, cuda-gdb 3.1 doesn’t run on my workstation, so I was using cuda-gdb 8.

Moral of the story: Make sure your cuda-gdb version matches the linked cuda libraries!

After I recompiled the application with cuda 8, the error went away, and I could happily try to find other errors.