When they are like that a fast retrieve will some times entice them to bite. We were fishing off virginia for shark and a couple showed up and the only thing that worked was a fast retrieve as if the bait wanted to get away.
Many times, I have found that Cobia up here will not bite anything. This is especially true when I find them in clean water. I have had no problems hooking numerous cobia close to shore in dirty green water, but when I find them further offshore in clean blue/green water, they seem to ignore anything thrown at them. They act a lot like dolphin. Sometimes you can pull up to some flotsam and throw anything at them and they'll fight each other for it. Other times, they'll ignore everything in the box. Last year, I had most of my luck on whole clams. That will also attract the triggers (and sometimes AJ's and dolphin) if they are around.