I fish there quite a bit, try 2nd beach, it's south of the pool. About a 10 min walk from the pool. There's a deep water point there.
You can drive there with a car, it's one of the back alleys down to the beach.
You can check the pics out under the accomodation forum, entitled Mtwalume.. I've got a cottage down there that I rent out in Mtwalume holiday resort.
There are pics of the point and 2nd beach.