The 'safest' way to do it is to have a physical switch box such as click here
However, you may be able to get away with using a simple 2-way splitter. The type will depend on whether the audio cables use stereo jack plugs, e.g. click here
or whether they use phono plugs, in which case of pair of these click here might do it (one each for the L & R channels).
Health Warning: Make sure all equipment is switched off before cabling up (just in case of static damage between boxes). HTH.