We have carried out KHL band high resolution imaging and H band imaging-polarimetry of the Red Rectangle nebula using CIAO and 36 element AO mounted on the 8.2m Subaru telescope. HK band images show a X-shape structure close to 0.1 inch and 2 lobes with separation of 0.15 inch at the north and the south. Our L band image show a small clump and its position is 0.1 arcsec east from the center of the southern lobe. The polarization map shows roughly centrosymmetric vector pattern and the center of the pattern is consistent with that of 2 lobes. There is scatter of the vector pattern at approximately 0.1 inch east from the southern lobe and a local minimum in the degree of polarization. These results can explain that the primary star HD44179 is at 0.1 inch east from the southern lobes and the dominant illumination source is a M type star at the center of the nebula.