“Monde” is both the word “world” and a synonym for “a group of persons”. The second meaning is an older formula for people you are used to or people who work for you.
-> “Je connais mon monde” == “I know my people”