In it's purest form, fill light shouldn't need to be softened. (I assume you are trying to soften the light, not just diffuse it).
Your key/main light is the one that is used to 'model' your subject....to give it form & depth by creating highlights & shadows. The fill light, on the other hand, shouldn't cause any shadows. It's purpose it to add light to the scene, in order to light the areas that the main light left in shadow....thereby allowing you to control the ratio between the highlights & the shadows.
In other words, you can put the fill light right on the camera axis, thus creating a 'flat' fill light...and it doesn't really have to be a soft light because any shadows that it may create, will be behind the subject where the camera can't see.
If you have the fill light off of the camera axis, it opens the possibility of it casting shadows, which you don't want. So in this case, you want to keep a close watch on the position and the output (and thus the ratio). But yes, if you are going to have the fill light off axis, it can help to have it softened, so that any shadows it may cause, will be softer/smoother.
Lastly, just putting something over the flash head will diffuse the light, but not necessarily soften it. To soften a light, you need to move it closer to the subject or make it physically bigger. That's why we use umbrellas, to enlarge the light source. If you just put a plastic cup over the flash, it may get a tiny bit softer, but really the only softening you get, is if it bounces around the room and reflects off of walls or the ceiling. So to soften your fill light, why not get another umbrella?