Newspapers. Birth Records are not publicly available for this time period. Assuming you know the names of the parents (specifically the father's surname) and an approximate area of where they were living at that time, check all newspapers in that area for any child born with that surname. You might also look for an obituary for one or both parents in case the children were listed as pre-deceased or survivors.
2010 Update: Birth Records are now available up to 1912 (1913 in May 2011) and the 1911 census is also available.