Understanding HDFS cluster FSImage image and looking its contents

FSImage in HDFS Cluster:

  • The HDFS namespace is stored by the NameNode.
  • The entire file system namespace, including the mapping of blocks to files and file system properties, is stored in a file called the FsImage.
  • The FsImage is stored as a file in the NameNode’s local file system too.
  • The NameNode keeps an image of the entire file system namespace and file Blockmap in memory.
  • Reads at startup only and keeps it during the life of start
    • RAM Dependency
    • What about “One Large File” or  “1,000,000 small files”

Step 1: First save the fsimage to local file system (Note: You cannot save the fsimage to HDFS.)

Step 2: Converting saved image to human readable file using –oiv option

Step 3: Now you can open the file in your favorite editor or you can try $head or $tail quickly to peek into.

fsimage

Advertisements

3 thoughts on “Understanding HDFS cluster FSImage image and looking its contents

  1. I drop a leave a response each time I like a post on a site or
    I have something to add to the discussion. Usually it is caused by the
    passion displayed in the article I browsed.
    And on this post Understanding HDFS cluster FSImage image and looking its contents | Cloud
    Celebrity (All about Cloud). I was actually excited enough to drop a leave a responsea response :
    -P I actually do have a couple of questions for you if it’s okay. Is it simply me or does it look like a few of the comments look like they are left by brain dead visitors? 😛 And, if you are posting at other online social sites, I’d like to keep
    up with you. Could you make a list the complete urls of all your community sites like your linkedin profile, Facebook page or
    twitter feed?

    Like

  2. Great post. I was checking continuously this weblog and
    I’m inspired! Extremely helpful info specifically the final part 🙂 I take care of such information much.
    I used to be seeking this certain information for a long time.
    Thanks and best of luck.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s