I may have found a workable solution.
I made a 5x5 swivel that was able to support a heavy load (150 blocks and 40+ braces) from one side.
CONSTRUCTION DETAILS
BASE PLATES: Make two walls 5 wide, 5 high, and 1 thick, facing each other with a 2 block space between them. These two plates are connected with bracing to form a single 5x5x3 unit. Both plates need to be heavily reinforced. They will serve as the races for the rollers.
Note: In the video you see some braces falling off. This is mostly from the points where the base attaches to the bracing coming up from the ground. I heavily braced the plates, but only on the outside faces. If I had reinforced the inside face and had directly attached the base plate to a 5x5 surface instead of holding it up by a few points, I believe it would have been much more stable.
ROLLER HUB: On the facing side of each wall, place an un-powered wheel on the center block. Place a block between the two wheels, and brace all sides of the block to each of the wheel hubs. you should now have a very sturdy and freely turning block between the two walls.
Note: This hub is based on the wheel-block-wheel swivel I shared earlier. Wheel attachments are very strong all by themselves, and when you divide a load between TWO of those and they each have a 5-brace connection to the central block (I connect it every face of the block except the one right up against the wheel--I can't seem to get that one), it's pretty durable.
ROLLERS: On one face of the hub, place 3 blocks, a wheel, and a 4th block. Delete the 3 inside blocks and the wheel leaving the 4th block (1/2 space beyond the edge of the wall). Place a block where the wheel was (against the inside face of the 4th block) and delete the 4th block. You now have a block half centered between the gap in the wall and sticking out 1/2 block. On the inside face of this block, place a stack of 3 un-powered wheels. Now brace from the outside face of the block to the hub. Repeat this for the other 3 faces of the hub. You will end up with a freely spinning + shaped assembly of rollers (12 wheels total) all connected to the central hub.
Note: It is the rollers and the hub together that will bear the weight. Instead of a single point of attachment to the base (which would either break or tear apart what it's attached to), the rollers+hub distribute the load at up to 14 freely moving points of contact across two 5x5 surfaces. Under ideal conditions, you can have 6 wheels pressing against the inside of the bottom base plate, 6 pressing against the inside of the top base plate, and the hub pressing at the center of each plate.
ATTACHMENT PLATE: The face that attaches to the load can be another 5x5 plate, or, as I did in this model, just a 5x5 brace frame. Either way, you will have to leave a small gap between it and the top base plate so that they can turn past one another. Start by placing a wheel in the center of the top base plate. Place a block on the wheel, then delete the wheel. You now have a block 1/2 space away from the base. Build up your attachment plate off of that. Reinforce it well.
Note: If you have an especially heavy load, I suggest using a full plate instead of a frame as I did. I found that trying to connect at only 4 or 5 points led to the load tearing under its own weight! With a plate, you can spread that force out across a wider area.
PIVOT: This is where you take advantage of the fact that the game treats braces as non-solid objects. Since they can just pass through other blocks, we can directly connect the attachment face to the rollers/hub without having to direct an axle though a hole in the center of the top plate. Not only would that be a pain, it would also add a lot more flex into the connection. As it is now, brace the blocks at the ends of the rollers straight up to the edges of the attachment plate and to the center as well. Connect the hub block to all the same points. Add any side/cross bracing you see a need for, just be sure to PASS THROUGH the top base plate, and not accidentally connect to it.
COMPLETION: Now you have the completed swivel. Attach it very firmly to the side of a heavy structure or brace it heavily as I did in the video. There will be a lot of torque. Now attach your load to the attachment plate and let it swing.
In the video I used a load of 150 blocks and about 50 braces (the load kept coming apart... especially at the attachment points). Even under that load, the base was braced well enough that there was no rotation toward the load, and the roller/attachment assembly flexed about 5-10 degrees mostly due to the play between the rollers and the base plates. Also: I did have to run it at about 5-15% speed to keep the physics from bugging out. Not sure if that is the game, my computer, or the model itself. I recorded the video at 6% and then increased the playback speed.
This setup is pretty sturdy as an off-balance pivot, but with two of these facing each other (one connecting to either side of your lever) I can't even imagine how much weight it could handle!
Attachments: