Results 1 to 5 of 5
  1. #1
    hkansal is offline Member
    Join Date
    Oct 2008
    Posts
    7
    Rep Power
    0

    Default parsing/storing large text data

    Hi All,

    I have seen a lot of pre-cooked java forum packages on the net but I would like to code my own and hence need some support. I have expertise in databases and have designed the required schema. Now to start I would like to have some help on this issue:

    In forums, eg this one, users may reply with long text inputs. What would be the best way to store them? One option is to use a Text field in the database which stores 65K characters. Another is to convert the large text to binary and store.

    Pls tell if there is any other option, else how shall I about accomplishing the second one?

    Thank You.

  2. #2
    Eranga's Avatar
    Eranga is offline Moderator
    Join Date
    Jul 2007
    Location
    Colombo, Sri Lanka
    Posts
    11,372
    Blog Entries
    1
    Rep Power
    19

    Default

    I don't know there is any other way is available. Over the years I'm using Text in SQL. It's much better so far so good. Even though data converted into binary values, I think no need to convert them into binaries within our code.

    What you are trying to do with this. Performance wise I don't see any issues working with Text.

  3. #3
    Nicholas Jordan's Avatar
    Nicholas Jordan is offline Senior Member
    Join Date
    Jun 2008
    Location
    Southwest
    Posts
    1,018
    Rep Power
    7

    Post is server to be remote?

    Quote Originally Posted by hkansal View Post
    Hi All, I have seen a lot of pre-cooked java forum packages on the net but I would like to code my own and hence need some support. I have expertise in databases and have designed the required schema. Now to start I would like to have some help on this issue:
    Can you provide some detail as to whether these databases are commercial in nature or student work, and is the database to be loaded onto a server remotely or can you code the whole thing directly in the application. The reason I ask this is expertise in databases and would like to have some help on this issue taken together suggests to me you will be taking a pure database approch. That raises in my mind an obvious ambiguity along the lines of one can write a file very simply with code, and storing data as a string or as binary is not a paticularly difficult design issue. So it is, politely intended, what gives here. (?)

    Quote Originally Posted by hkansal View Post
    In forums, eg this one, users may reply with long text inputs. What would be the best way to store them? One option is to use a Text field in the database which stores 65K characters. Another is to convert the large text to binary and store.
    Compression, through available Java classes is available but given the massive storage available at commodity pricing I would focus on "what makes sense to you" before I would look to established design for preliminary concepting. I see a lot of work in Marty Hall's Core Servlets book that may provide a better starting place than considering the matter directly, there are some issues related to scaling that may be of profound consequence later. If this will ever run on a distributed Tier-1 arena, that would be well worth the effort later.

    Quote Originally Posted by hkansal View Post
    Pls tell if there is any other option, else how shall I about accomplishing the second one?
    There are plenty of options, as long as you stay with Java, the competitors are not True Beans.
    Introduction to Programming Using Java.
    Cybercartography: A new theoretical construct proposed by D.R. Fraser Taylor

  4. #4
    hkansal is offline Member
    Join Date
    Oct 2008
    Posts
    7
    Rep Power
    0

    Default

    The database is designed to hold data in a recursive manner, self-referencing keys leading to space optimization. The doubt I had was raised because if text is saved as text itself,
    1. it occupies more space.
    2. there is a limit to the number of characters. Posts in a forum may exceed that.
    also there is a feature in all databases to store large objects, and objects in binary form. this method i can use to also save images in the db itself(though i am not planning to do that).

    Kindly advise me the good techniques to go about solving my problem.

  5. #5
    hkansal is offline Member
    Join Date
    Oct 2008
    Posts
    7
    Rep Power
    0

    Default

    And yes, this is a student work and but the whole db can be coded in the application itself or uploaded remotely, both.

Similar Threads

  1. Replies: 0
    Last Post: 04-22-2008, 11:57 AM
  2. [SOLVED] How to Extract Data From this text file?
    By jazz2k8 in forum New To Java
    Replies: 31
    Last Post: 04-18-2008, 10:45 AM
  3. Storing data permanently
    By shaungoater in forum New To Java
    Replies: 2
    Last Post: 03-10-2008, 04:18 PM
  4. Displaying data into text area
    By abhiN in forum New To Java
    Replies: 1
    Last Post: 01-22-2008, 10:30 AM
  5. Storing Data
    By Khorod in forum New To Java
    Replies: 1
    Last Post: 08-03-2007, 05:48 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •