I have the same MacPro and have a similar situation. I've been using a GTX980Ti with varying luck. For the most part is seems pretty well behaved using the Nvidia Web Drivers and CUDa as Justin suggests, but every once in a while formZ and it seem to disagree (usually at critical deadline times) and I wind up pulling the card and throwing my old Radeons back in. Even with the Radeons though I am pretty impressed with the speed and results of Thea when using its biased engines. At HD res this render was about 45 minutes. At 4000x2250 it was about 1 hr 45 minutes. Scene is completely lit by the candles and some IBL flames in the fireplace. I've been working on and off with the Thea Beta since it came out and I'm getting more comfortable with it and am extremely happy.
The exact Nvidia problems are really hard to track down. As I said it is stable most of the time and I'm thrilled with the card when it is working. I'm still running Yosemite, and perhaps it is more stable with El Capitain but I hate doing OS updates unless I have a good chunk of time to sort out the problems, and there are always problems.
Heartily endorse Thea for formZ. Each new release of the beta gets better and better.
Edited by DMclean, 13 July 2016 - 10:21 AM.